-- Logs begin at Tue 2021-05-25 15:01:54 UTC, end at Tue 2021-05-25 16:23:02 UTC. -- May 25 15:54:18.607327 dv-cinder-786384-6-iscsi systemd[1]: Started Devstack devstack@q-svc.service. May 25 15:54:19.594731 dv-cinder-786384-6-iscsi neutron-server[99435]: Beware of the coming changes in the project packaging May 25 15:54:19.594731 dv-cinder-786384-6-iscsi neutron-server[99435]: https://github.com/svinota/pyroute2/discussions/786 May 25 15:54:21.064167 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.common.config [-] Logging enabled! May 25 15:54:21.064809 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev240 May 25 15:54:21.064918 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=99435) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 25 15:54:21.068677 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.common.config [-] Logging enabled! May 25 15:54:21.068933 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.common.config [-] /usr/local/bin/neutron-server version 18.1.0.dev240 May 25 15:54:21.069238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] command line: /usr/local/bin/neutron-server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini {{(pid=99435) setup_logging /opt/stack/neutron/neutron/common/config.py:112}} May 25 15:54:21.070172 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 15:54:21.070405 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] Configuration options gathered from: {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 15:54:21.070776 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 15:54:21.071045 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 15:54:21.071268 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ================================================================================ {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 15:54:21.071585 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] allow_bulk = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.071874 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] allow_overlapping_ips = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.072166 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] api_extensions_path = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.072394 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] api_paste_config = api-paste.ini {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.072682 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] api_workers = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.072973 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] auth_strategy = keystone {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.073252 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] backlog = 4096 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.073687 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] base_mac = fa:16:3e:00:00:00 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.073987 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] bind_host = 0.0.0.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.074270 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] bind_port = 9696 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.074547 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] client_socket_timeout = 900 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.074829 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] config_dir = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.075086 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.075364 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] config_source = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.075649 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] control_exchange = neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.075942 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] core_plugin = ml2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.076158 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] debug = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.076437 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] default_availability_zones = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.076865 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.077284 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] dhcp_agent_notification = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.077558 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] dhcp_lease_duration = 86400 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.077843 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] dns_domain = openstacklocal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.078446 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] enable_traditional_dhcp = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.078556 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] external_dns_driver = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.078758 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] filter_validation = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.079074 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] global_physnet_mtu = 1500 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.079412 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] host = dv-cinder-786384-6-iscsi {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.079745 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] http_retries = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.079972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] instance_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.080255 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.080602 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ipam_driver = internal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.080854 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ipv6_pd_enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.081104 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_config_append = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.081507 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.081658 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_dir = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.081925 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.082197 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_rotate_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.082483 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_rotate_interval_type = days {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.082768 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] log_rotation_type = none {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.083100 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.083319 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.083540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.083914 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] max_dns_nameservers = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] max_header_line = 16384 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] max_logfile_count = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] max_logfile_size_mb = 200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085546 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] max_subnet_host_routes = 20 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085702 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] network_link_prefix = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.085983 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] notify_nova_on_port_data_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.086363 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] notify_nova_on_port_status_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.086573 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] pagination_max_limit = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.086821 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] periodic_fuzzy_delay = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.087120 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] periodic_interval = 40 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.087390 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] publish_errors = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.087658 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rate_limit_burst = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.087938 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rate_limit_except_level = CRITICAL {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.088207 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rate_limit_interval = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.088480 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] retry_until_window = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.088751 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rpc_response_max_timeout = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.089035 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rpc_state_report_workers = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.089301 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] rpc_workers = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.089571 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] send_events_interval = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.089876 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.090154 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] setproctitle = on {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.090431 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] state_path = /opt/stack/data/neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.090715 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] syslog_log_facility = LOG_USER {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.091092 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] tcp_keepidle = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.091294 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.091561 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_eventlog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.091834 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_journal = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.092102 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_json = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.092371 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.092640 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_stderr = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.092920 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] use_syslog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.093189 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] vlan_transparent = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.093474 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] watch_log_file = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.093729 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] wsgi_default_pool_size = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.093997 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] wsgi_keep_alive = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.094276 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:21.094574 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_concurrency.disable_process_locking = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.094893 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.095189 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.connection_string = messaging:// {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.095471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.095765 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.es_doc_type = notification {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.096043 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.es_scroll_size = 10000 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.096365 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.es_scroll_time = 2m {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.096647 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.filter_error_trace = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.096936 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.hmac_keys = SECRET_KEY {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.097228 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.sentinel_service_name = mymaster {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.097515 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.socket_timeout = 0.1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.097792 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] profiler.trace_sqlalchemy = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.098081 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.enforce_new_defaults = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.098357 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.enforce_scope = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.098651 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.policy_default_rule = default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.098979 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.099273 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.099562 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.099836 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.100192 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.100379 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.100656 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.100948 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.101245 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.auth_type = password {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.101525 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.101792 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.102072 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.102366 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.102644 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.102940 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.103220 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.live_migration_events = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.103510 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.region_name = RegionOne {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.103790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.104064 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] nova.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.104347 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.104627 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.104900 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.105172 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.105451 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.105738 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.106072 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.106289 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.106565 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.106843 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.107139 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] placement.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.107782 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.107782 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.108119 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.108399 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.108685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.108959 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.connect_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.109231 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.connect_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.109511 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.enable_notifications = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.109781 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.endpoint_override = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.110055 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.110330 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.interface = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.110599 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.110906 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.max_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.111168 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.min_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.111429 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.111698 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.service_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.112483 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.service_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.112850 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.113126 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.status_code_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.113400 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.status_code_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.113673 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.113947 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.valid_interfaces = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.114219 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ironic.version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.114507 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] service_providers.service_provider = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.114806 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.115122 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.115403 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.115683 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.115961 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.116237 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.116513 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.117046 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.117324 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.117597 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.117888 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.118173 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.118459 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.118738 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.119134 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.119393 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.119630 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.119875 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.120153 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.120432 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.120719 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.121006 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.121305 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.121598 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.121890 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.122183 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_rabbit.ssl_version = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.122475 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_notifications.driver = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.122801 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_notifications.retry = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.123205 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.123381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] oslo_messaging_notifications.transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:21.123600 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.common.config [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 15:54:21.123849 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo.service.wsgi [-] Loading app neutron from /etc/neutron/api-paste.ini {{(pid=99435) load_app /usr/local/lib/python3.8/dist-packages/oslo_service/wsgi.py:349}} May 25 15:54:21.468512 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" acquired by "neutron_lib.plugins.directory._create_plugin_directory" :: waited 0.000s {{(pid=99435) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:21.468780 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [-] Lock "plugin-directory" released by "neutron_lib.plugins.directory._create_plugin_directory" :: held 0.001s {{(pid=99435) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:21.469068 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [-] Lock "manager" acquired by "neutron.manager.NeutronManager._create_instance" :: waited 0.000s {{(pid=99435) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:21.469540 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [-] Loading core plugin: ml2 May 25 15:54:21.859239 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.859527 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.859799 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.860977 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.861249 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.861512 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.896653 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.896927 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.897196 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.910624 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.910926 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.911198 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.940386 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.940666 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.940938 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.977985 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.978263 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:21.978530 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.209354 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.210001 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.210001 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment precommit_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.210220 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: network precommit_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.210899 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: segment before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.336088 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.336405 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: agent before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.336685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.336960 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.337221 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.337537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.337779 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment precommit_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.338029 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.338284 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.338567 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.338823 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > network precommit_delete 0 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.339116 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port provisioning_complete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.339379 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.339631 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.339898 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > agent after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.340162 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.341696 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > subnet precommit_delete 0 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.341992 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.342258 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.342555 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.342938 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.343203 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.343459 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.343752 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.343998 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.344245 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.344648 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: network configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.344887 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: port configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.345126 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnet configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.345363 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: subnetpool configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.345600 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.345835 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [-] Tracking information for resource: security_group_rule configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:22.346202 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] May 25 15:54:22.350104 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_flat [-] Allowable flat physical_network names: ['physnet1'] May 25 15:54:22.354008 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_local [-] ML2 LocalTypeDriver initialization complete May 25 15:54:22.357837 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_vlan [-] Network VLAN ranges: OrderedDict([('physnet1', [(900, 949)])]) May 25 15:54:22.359576 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan'] May 25 15:54:22.360021 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve', 'gre', 'local', 'vlan', 'vxlan']) May 25 15:54:22.360524 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['vlan'] May 25 15:54:22.360730 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security'] May 25 15:54:22.364400 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security'] May 25 15:54:22.364648 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security'] May 25 15:54:22.364998 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['openvswitch', 'hyperv'] May 25 15:54:22.392404 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.openvswitch.mech_driver.mech_openvswitch [-] Open vSwitch agent's supported_vnic_types: ['normal', 'direct', 'smart-nic', 'vdpa'] May 25 15:54:22.392853 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > qos_plugin after_init 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.393185 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.qos.drivers.openvswitch.driver [-] Open vSwitch QoS driver registered {{(pid=99435) register /opt/stack/neutron/neutron/services/qos/drivers/openvswitch/driver.py:82}} May 25 15:54:22.393548 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > logging-plugin after_init 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.394924 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.logapi.drivers.openvswitch.driver [-] Open vSwitch logging driver registered {{(pid=99435) register /opt/stack/neutron/neutron/services/logapi/drivers/openvswitch/driver.py:69}} May 25 15:54:22.395215 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['openvswitch', 'hyperv'] May 25 15:54:22.395461 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['openvswitch', 'hyperv'] May 25 15:54:22.398727 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > floatingip before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.399060 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [-] Subscribe: > port before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:22.462267 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' May 25 15:54:22.462448 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete May 25 15:54:22.462648 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' May 25 15:54:22.463054 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 1000)] May 25 15:54:22.494832 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_db.sqlalchemy.engines [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=99435) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 15:54:22.977122 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing driver for type 'gre' May 25 15:54:22.977673 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] gre ID ranges: [(1, 1000)] May 25 15:54:23.204961 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing driver for type 'local' May 25 15:54:23.205302 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing driver for type 'vlan' May 25 15:54:23.313888 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_vlan [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] VlanTypeDriver initialization complete May 25 15:54:23.314522 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing driver for type 'vxlan' May 25 15:54:23.314662 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.drivers.type_tunnel [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] vxlan ID ranges: [(1, 1000)] May 25 15:54:23.605321 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing extension driver 'port_security' May 25 15:54:23.605567 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.extensions.port_security [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] PortSecurityExtensionDriver initialization complete May 25 15:54:23.605899 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing mechanism driver 'openvswitch' May 25 15:54:23.606224 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing mechanism driver 'hyperv' May 25 15:54:23.613246 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.plugins.ml2.plugin [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] neutron.plugins.ml2.plugin.Ml2Plugin method _start_rpc_notifiers called with arguments () {} {{(pid=99435) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:23.614039 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.614417 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.614831 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.615234 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.615633 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.615972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.616349 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.616942 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.617280 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.617651 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.617994 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.618330 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.618709 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group_rule after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.619073 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group_rule after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623091 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > security_group_rule after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623091 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > address_group after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623091 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > address_group after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623091 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > address_group after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623091 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.plugins.ml2.ovo_rpc [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] ML2 OVO RPC backend initialized. {{(pid=99435) __init__ /opt/stack/neutron/neutron/plugins/ml2/ovo_rpc.py:160}} May 25 15:54:23.623471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > networks before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623780 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623780 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > ports before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.623780 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.624010 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnets before_response 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.624149 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.624486 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.624814 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.625147 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.625487 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.625819 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.626152 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.626495 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.626856 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > subnet after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.627993 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.plugin [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Modular L2 Plugin initialization complete May 25 15:54:23.628321 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading services supported by the core plugin {{(pid=99435) _load_services_from_core_plugin /opt/stack/neutron/neutron/manager.py:168}} May 25 15:54:23.628652 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.plugins.ml2.managers [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Got port-security extension from driver 'port_security' May 25 15:54:23.629115 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.extensions.vlantransparent [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Disabled vlantransparent extension. May 25 15:54:23.629601 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading service plugins: ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin', 'auto_allocate', 'tag', 'timestamp', 'network_ip_availability', 'flavors', 'revisions'] {{(pid=99435) _load_service_plugins /opt/stack/neutron/neutron/manager.py:194}} May 25 15:54:23.629913 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: router May 25 15:54:23.667868 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.668268 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_create 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.668634 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router before_create 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.668991 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.669360 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.669712 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_delete 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.670064 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > floatingip after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.670428 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_create 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.670766 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.671144 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.671488 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_gateway after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.671838 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.672240 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.672610 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.672949 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.673290 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: subnet after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.673629 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: subnetpool_address_scope after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.673979 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > port precommit_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.674357 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_create 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.674696 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.675070 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.675427 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router after_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.675767 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_delete 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.676110 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_create 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.676469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router_interface after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.676811 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_update 55549900 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.677181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > network after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.678076 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Tracking information for resource: router configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:23.678404 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Tracking information for resource: floatingip configured {{(pid=99435) set_tracked_resource /opt/stack/neutron/neutron/quota/resource_registry.py:205}} May 25 15:54:23.683593 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.db.l3_agentschedulers_db [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Skipping period L3 agent status check because automatic router rescheduling is disabled. May 25 15:54:23.684176 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.684538 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.684886 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.685235 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: port after_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.686049 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router before_create 55550100 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.686406 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_delete 55550100 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.686761 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_create 55550100 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.687146 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > router precommit_update 55550100 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.692242 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.provider_configuration [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service providers = [] {{(pid=99435) parse_service_provider_opt /opt/stack/neutron/neutron/services/provider_configuration.py:171}} May 25 15:54:23.692600 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.db.servicetype_db [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Adding provider configuration for service L3_ROUTER_NAT {{(pid=99435) add_provider_configuration /opt/stack/neutron/neutron/db/servicetype_db.py:42}} May 25 15:54:23.695522 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.service_base [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded 'neutron.services.l3_router.service_providers.dvrha.DvrHaDriver' provider for service L3_ROUTER_NAT {{(pid=99435) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 25 15:54:23.695866 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.service_base [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded 'neutron.services.l3_router.service_providers.dvr.DvrDriver' provider for service L3_ROUTER_NAT {{(pid=99435) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 25 15:54:23.696201 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.service_base [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded 'neutron.services.l3_router.service_providers.ha.HaDriver' provider for service L3_ROUTER_NAT {{(pid=99435) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 25 15:54:23.697059 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.services.service_base [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded 'neutron.services.l3_router.service_providers.single_node.SingleNodeDriver' provider for service L3_ROUTER_NAT {{(pid=99435) load_drivers /opt/stack/neutron/neutron/services/service_base.py:49}} May 25 15:54:23.697425 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.services.service_base [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Default provider is not specified for service type L3_ROUTER_NAT May 25 15:54:23.697976 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded L3_ROUTER_NAT plugin. Description: L3 Router Service Plugin for basic L3 forwarding between (L2) Neutron networks and access to external networks via a NAT gateway. {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.698293 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: neutron.services.metering.metering_plugin.MeteringPlugin May 25 15:54:23.709420 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded METERING plugin. Description: METERING {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.709748 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: auto_allocate May 25 15:54:23.715379 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: network precommit_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.715737 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: network precommit_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.716248 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded auto-allocated-topology plugin. Description: Auto Allocated Topology - aka get me a network. {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.716561 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: tag May 25 15:54:23.723772 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded TAG plugin. Description: Tag support {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.724100 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: timestamp May 25 15:54:23.727949 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded timestamp plugin. Description: Adds timestamps to Neutron resources with standard attributes {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.728268 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: network_ip_availability May 25 15:54:23.733473 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.733841 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.734196 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.735093 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded network-ip-availability plugin. Description: Provides IP availability data for each network and subnet. {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.735265 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: flavors May 25 15:54:23.740342 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded FLAVORS plugin. Description: Neutron Flavors and Service Profiles manager plugin {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.740669 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading Plugin: revisions May 25 15:54:23.745122 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Successfully loaded revision_plugin plugin. Description: Adds revision numbers to resources. {{(pid=99435) _create_and_add_service_plugin /opt/stack/neutron/neutron/manager.py:229}} May 25 15:54:23.745506 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Lock "manager" released by "neutron.manager.NeutronManager._create_instance" :: held 2.276s {{(pid=99435) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:23.746237 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] get_extension_paths = OrderedDict([('/opt/stack/neutron/neutron/extensions', 1)]) {{(pid=99435) get_extensions_path /opt/stack/neutron/neutron/api/extensions.py:635}} May 25 15:54:23.746570 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Initializing extension manager. May 25 15:54:23.747114 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: __init__.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.747453 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: __pycache__ {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.747785 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: _standard_attr_segment_lib.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.748115 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: address_group.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.748872 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Address group" alias="address-group" description="Support address group" updated="2020-07-15T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.749439 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: address-group May 25 15:54:23.749756 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: address_scope.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.750451 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Address scope" alias="address-scope" description="Address scopes extension." updated="2015-07-26T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.750978 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: address-scope May 25 15:54:23.751296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: admin_state_down_before_update.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.752200 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Enforce Router's Admin State Down Before Update Extension" alias="router-admin-state-down-before-update" description="Ensure that the admin state of a router is DOWN (admin_state_up=False) before updating the distributed attribute" updated="2019-07-02T15:56:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.753225 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: router-admin-state-down-before-update May 25 15:54:23.753542 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: agent.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.754229 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="agent" alias="agent" description="The agent management extension." updated="2013-02-03T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.754741 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: agent May 25 15:54:23.755081 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: agent_resources_synced.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.755972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Agent's Resource View Synced to Placement" alias="agent-resources-synced" description="Stores success/failure of last sync to Placement" updated="2018-12-19T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.756469 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: agent-resources-synced May 25 15:54:23.756786 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: allowedaddresspairs.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.758570 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Allowed Address Pairs" alias="allowed-address-pairs" description="Provides allowed address pairs" updated="2013-07-23T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.759106 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: allowed-address-pairs May 25 15:54:23.759428 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: auto_allocated_topology.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.760490 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Auto Allocated Topology Services" alias="auto-allocated-topology" description="Auto Allocated Topology Services." updated="2016-01-01T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.760997 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: auto-allocated-topology May 25 15:54:23.761322 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: availability_zone.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.762002 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Availability Zone" alias="availability_zone" description="The availability zone extension." updated="2015-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.762505 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: availability_zone May 25 15:54:23.762824 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: availability_zone_filter.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.763740 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Availability Zone Filter Extension" alias="availability_zone_filter" description="Add filter parameters to AvailabilityZone resource" updated="2018-06-22T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.764240 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: availability_zone_filter May 25 15:54:23.764557 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: data_plane_status.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.765444 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port data plane status extension" alias="data-plane-status" description="Expose status of underlying data plane" updated="2017-01-24T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.765939 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension data-plane-status not supported by any of loaded plugins May 25 15:54:23.766253 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: default_subnetpools.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.767258 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Default Subnetpools" alias="default-subnetpools" description="Provides ability to mark and use a subnetpool as the default." updated="2016-02-18T18:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.767752 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: default-subnetpools May 25 15:54:23.768090 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: dhcpagentscheduler.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.769509 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="DHCP Agent Scheduler" alias="dhcp_agent_scheduler" description="Schedule networks among dhcp agents" updated="2013-02-07T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.770013 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: dhcp_agent_scheduler May 25 15:54:23.770330 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: dns.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.771251 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="DNS Integration" alias="dns-integration" description="Provides integration with DNS." updated="2015-08-15T18:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.771749 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension dns-integration not supported by any of loaded plugins May 25 15:54:23.772060 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: dns_domain_ports.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.772967 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="dns_domain for ports" alias="dns-domain-ports" description="Allows the DNS domain to be specified for a network port." updated="2017-04-24T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.773461 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension dns-domain-ports not supported by any of loaded plugins May 25 15:54:23.773820 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: dvr.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.774431 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Distributed Virtual Router" alias="dvr" description="Enables configuration of Distributed Virtual Routers." updated="2014-06-1T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.774969 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: dvr May 25 15:54:23.775286 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: empty_string_filtering.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.776171 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Empty String Filtering Extension" alias="empty-string-filtering" description="Allow filtering by attributes with empty string value" updated="2018-05-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.776672 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: empty-string-filtering May 25 15:54:23.776984 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: expose_l3_conntrack_helper.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.777873 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Expose CT target rules for conntrack helper" alias="expose-l3-conntrack-helper" description="Expose allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.778362 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension expose-l3-conntrack-helper not supported by any of loaded plugins May 25 15:54:23.778677 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: expose_port_forwarding_in_fip.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.779606 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Expose Floating IP port forwardings" alias="expose-port-forwarding-in-fip" description="Expose port forwarding fields in floating IP response" updated="2018-06-28T16:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.780093 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension expose-port-forwarding-in-fip not supported by any of loaded plugins May 25 15:54:23.780409 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: external_net.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.781294 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron external network" alias="external-net" description="Adds external network attribute to network resource." updated="2013-01-14T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.781796 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: external-net May 25 15:54:23.782110 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: extra_dhcp_opt.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.783039 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Extra DHCP options" alias="extra_dhcp_opt" description="Extra options configuration for DHCP. For example PXE boot options to DHCP clients can be specified (e.g. tftp-server, server-ip-address, bootfile-name)" updated="2013-03-17T12:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.783535 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: extra_dhcp_opt May 25 15:54:23.783879 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: extraroute.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.784768 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Extra Route" alias="extraroute" description="Extra routes configuration for L3 router" updated="2013-02-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.785267 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: extraroute May 25 15:54:23.785582 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: extraroute_atomic.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.786739 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Atomically add/remove extra routes" alias="extraroute-atomic" description="Edit extra routes of a router on server side by atomically adding/removing extra routes" updated="2019-07-10T00:00:00+00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.787270 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: extraroute-atomic May 25 15:54:23.787596 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: filter_validation.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.788241 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Filter parameters validation" alias="filter-validation" description="Provides validation on filter parameters." updated="2018-07-04T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.788737 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: filter-validation May 25 15:54:23.789052 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: fip_pf_description.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.790145 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Floating IP Port Forwarding new attribute description" alias="floating-ip-port-forwarding-description" description="Add a description field to Port Forwarding rules" updated="2019-11-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.790643 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension floating-ip-port-forwarding-description not supported by any of loaded plugins May 25 15:54:23.790990 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: fip_port_details.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.791889 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Floating IP Port Details Extension" alias="fip-port-details" description="Add port_details attribute to Floating IP resource" updated="2018-04-09T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.792386 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: fip-port-details May 25 15:54:23.792703 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: flavors.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.793927 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Service Flavors" alias="flavors" description="Flavor specification for Neutron advanced services." updated="2015-09-17T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.794430 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: flavors May 25 15:54:23.794756 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: floating_ip_port_forwarding.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.796445 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Floating IP Port Forwarding" alias="floating-ip-port-forwarding" description="Allow port forwarding from floating IP port to an internal port" updated="2018-01-19T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.796955 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension floating-ip-port-forwarding not supported by any of loaded plugins May 25 15:54:23.797281 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: floatingip_pools.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.797972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Floating IP Pools Extension" alias="floatingip-pools" description="Provides a floating IP pools API." updated="2018-03-21T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.798473 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: floatingip-pools May 25 15:54:23.798794 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: ip_allocation.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.799760 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="IP Allocation" alias="ip_allocation" description="IP allocation extension." updated="2016-06-10T23:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.800256 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension ip_allocation not supported by any of loaded plugins May 25 15:54:23.800573 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: ip_substring_port_filtering.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.801481 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="IP address substring filtering" alias="ip-substring-filtering" description="Provides IP address substring filtering when listing ports" updated="2017-11-28T09:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.801979 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: ip-substring-filtering May 25 15:54:23.802296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l2_adjacency.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.803243 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="L2 Adjacency" alias="l2_adjacency" description="Display L2 Adjacency for Neutron Networks." updated="2016-04-12T16:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.803743 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension l2_adjacency not supported by any of loaded plugins May 25 15:54:23.804055 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.805032 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron L3 Router" alias="router" description="Router abstraction for basic L3 forwarding between L2 Neutron networks and access to external networks via a NAT gateway." updated="2012-07-20T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.805546 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: router May 25 15:54:23.805863 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3_conntrack_helper.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.808105 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="L3 Conntrack helper" alias="l3-conntrack-helper" description="Allow adding CT target rules for conntrack helper" updated="2019-04-04T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.808635 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension l3-conntrack-helper not supported by any of loaded plugins May 25 15:54:23.808966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3_ext_gw_mode.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.809886 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron L3 Configurable external gateway mode" alias="ext-gw-mode" description="Extension of the router abstraction for specifying whether SNAT should occur on the external gateway" updated="2013-03-28T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.810400 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: ext-gw-mode May 25 15:54:23.810726 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3_ext_ha_mode.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.811647 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="HA Router extension" alias="l3-ha" description="Adds HA capability to routers." updated="2014-04-26T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.812159 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: l3-ha May 25 15:54:23.812479 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3_flavors.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.813380 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Router Flavor Extension" alias="l3-flavors" description="Flavor support for routers." updated="2016-05-17T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.813879 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: l3-flavors May 25 15:54:23.814202 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3_port_ip_change_not_allowed.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.815641 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Prevent L3 router ports IP address change extension" alias="l3-port-ip-change-not-allowed" description="Prevent change of IP address for some L3 router ports" updated="2018-10-09T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.816142 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: l3-port-ip-change-not-allowed May 25 15:54:23.816462 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: l3agentscheduler.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.817485 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="L3 Agent Scheduler" alias="l3_agent_scheduler" description="Schedule routers among l3 agents" updated="2013-02-07T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.817988 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: l3_agent_scheduler May 25 15:54:23.818310 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: logging.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.819803 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Logging API Extension" alias="logging" description="Provides a logging API for resources such as security group and firewall." updated="2017-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.820322 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension logging not supported by any of loaded plugins May 25 15:54:23.820648 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: metering.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.821376 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Metering" alias="metering" description="Neutron Metering extension." updated="2013-06-12T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.821946 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: metering May 25 15:54:23.822266 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: metering_source_and_destination_fields.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.823366 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Metering with source and destination filtering" alias="metering_source_and_destination_fields" description="Neutron Metering extension that enables the use of source and destination IP prefixes to create metering label rules." updated="2020-08-21T08:12:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.823873 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: metering_source_and_destination_fields May 25 15:54:23.824192 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: multiprovidernet.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.825217 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Multi Provider Network" alias="multi-provider" description="Expose mapping of virtual networks to multiple physical networks" updated="2013-06-27T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.825724 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: multi-provider May 25 15:54:23.826044 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: netmtu.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.826974 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Network MTU" alias="net-mtu" description="Provides MTU attribute for a network resource." updated="2015-03-25T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.827477 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: net-mtu May 25 15:54:23.827796 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: netmtu_writable.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.828752 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Network MTU (writable)" alias="net-mtu-writable" description="Provides a writable MTU attribute for a network resource." updated="2017-07-12T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.829253 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: net-mtu-writable May 25 15:54:23.829571 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: network_availability_zone.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.830222 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Network Availability Zone" alias="network_availability_zone" description="Availability zone support for network." updated="2015-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.831373 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: network_availability_zone May 25 15:54:23.831716 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: network_ip_availability.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.834786 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Network IP Availability" alias="network-ip-availability" description="Provides IP availability data for each network and subnet." updated="2015-09-24T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.835347 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: network-ip-availability May 25 15:54:23.835670 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: network_segment_range.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.836456 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Network Segment Range" alias="network-segment-range" description="Provides support for the network segment range management" updated="2018-11-29T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.837024 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension network-segment-range not supported by any of loaded plugins May 25 15:54:23.837345 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: pagination.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.838316 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Pagination support" alias="pagination" description="Extension that indicates that pagination is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.838641 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: pagination May 25 15:54:23.838994 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: port_device_profile.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.839898 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port device profile" alias="port-device-profile" description="Expose the port device profile (Cyborg)" updated="2020-12-17T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.840403 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: port-device-profile May 25 15:54:23.840717 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: port_mac_address_regenerate.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.841902 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Port MAC address regenerate" alias="port-mac-address-regenerate" description="Network port MAC address regenerate" updated="2018-05-03T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.842493 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: port-mac-address-regenerate May 25 15:54:23.843059 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: port_numa_affinity_policy.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.843771 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port NUMA affinity policy" alias="port-numa-affinity-policy" description="Expose the port NUMA affinity policy" updated="2020-07-08T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.844336 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: port-numa-affinity-policy May 25 15:54:23.844668 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: port_resource_request.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.845639 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port Resource Request" alias="port-resource-request" description="Expose resource request to Port" updated="2018-05-08T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.846133 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension port-resource-request not supported by any of loaded plugins May 25 15:54:23.846478 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: portbindings.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.847951 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port Binding" alias="binding" description="Expose port bindings of a virtual port to external application" updated="2014-02-03T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.848736 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: binding May 25 15:54:23.849139 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: portbindings_extended.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.851007 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port Bindings Extended" alias="binding-extended" description="Expose port bindings of a virtual port to external application" updated="2017-07-17T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.851623 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: binding-extended May 25 15:54:23.852073 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: portsecurity.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.853027 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port Security" alias="port-security" description="Provides port security" updated="2012-07-23T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.853526 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: port-security May 25 15:54:23.853849 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: project_id.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.854817 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="project_id field enabled" alias="project-id" description="Extension that indicates that project_id field is enabled." updated="2016-09-09T09:09:09-09:09" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.855294 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: project-id May 25 15:54:23.855724 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: providernet.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.856758 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Provider Network" alias="provider" description="Expose mapping of virtual networks to physical networks" updated="2012-09-07T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.857475 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: provider May 25 15:54:23.857849 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.862078 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Quality of Service" alias="qos" description="The Quality of Service extension." updated="2015-06-08T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.862812 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos not supported by any of loaded plugins May 25 15:54:23.863207 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_bw_limit_direction.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.864527 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Direction for QoS bandwidth limit rule" alias="qos-bw-limit-direction" description="Allow to configure QoS bandwidth limit rule with specific direction: ingress or egress" updated="2017-04-10T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.865158 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-bw-limit-direction not supported by any of loaded plugins May 25 15:54:23.865487 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_bw_minimum_ingress.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.866604 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Ingress direction for QoS minimum bandwidth rule" alias="qos-bw-minimum-ingress" description="Allow to configure QoS minumum bandwidth rule with ingress direction." updated="2018-07-09T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.867531 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-bw-minimum-ingress not supported by any of loaded plugins May 25 15:54:23.867853 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_default.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.868753 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="QoS default policy" alias="qos-default" description="Expose the QoS default policy per project" updated="2017-041-06T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.869311 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-default not supported by any of loaded plugins May 25 15:54:23.869639 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_fip.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.870288 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Floating IP QoS" alias="qos-fip" description="The floating IP Quality of Service extension" updated="2017-07-20T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.870785 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-fip not supported by any of loaded plugins May 25 15:54:23.871137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_gateway_ip.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.872040 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Router gateway IP QoS" alias="qos-gateway-ip" description="The Router gateway IP Quality of Service extension" updated="2018-02-24T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.872545 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-gateway-ip not supported by any of loaded plugins May 25 15:54:23.872860 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_port_network_policy.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.873799 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="QoS port network policy ID" alias="qos-port-network-policy" description="Adds a the QoS network ID to the port definition" updated="2019-11-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.874299 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-port-network-policy not supported by any of loaded plugins May 25 15:54:23.874621 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_rule_type_details.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.875583 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Details of QoS rule types" alias="qos-rule-type-details" description="Expose details about QoS rule types supported by loaded backend drivers" updated="2017-06-22T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.876078 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-rule-type-details not supported by any of loaded plugins May 25 15:54:23.876395 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: qos_rules_alias.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.877936 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Quality of Service rules alias API" alias="qos-rules-alias" description="API to enable GET, PUT and DELETE operations on QoS policy rules without specifying policy ID" updated="2018-10-07T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.878442 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension qos-rules-alias not supported by any of loaded plugins May 25 15:54:23.878758 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: quotasv2.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.879849 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Quota management support" alias="quotas" description="Expose functions for quotas management per tenant" updated="2012-07-29T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.880169 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: quotas May 25 15:54:23.880492 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: quotasv2_detail.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.882233 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Quota details management support" alias="quota_details" description="Expose functions for quotas usage statistics per project" updated="2017-02-10T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.882569 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: quota_details May 25 15:54:23.882969 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: rbac.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.883968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="RBAC Policies" alias="rbac-policies" description="Allows creation and modification of policies that control tenant access to resources." updated="2015-06-17T12:15:12-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.884480 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: rbac-policies May 25 15:54:23.884877 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: rbac_address_group.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.885789 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Add address_group type to RBAC" alias="rbac-address-group" description="Add address_group type to network RBAC" updated="2021-01-20T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.886298 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: rbac-address-group May 25 15:54:23.886611 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: rbac_address_scope.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.887537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Add address_scope type to RBAC" alias="rbac-address-scope" description="Add address_scope type to RBAC" updated="2020-02-12T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.888038 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: rbac-address-scope May 25 15:54:23.888357 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: rbac_security_groups.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.889256 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Add security_group type to network RBAC" alias="rbac-security-groups" description="Add security_group type to network RBAC" updated="2019-02-14T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.889754 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: rbac-security-groups May 25 15:54:23.890072 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: rbac_subnetpool.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.890990 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Add subnetpool type to RBAC" alias="rbac-subnetpool" description="Add subnetpool type to RBAC" updated="2020-02-05T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.891494 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: rbac-subnetpool May 25 15:54:23.891809 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: revisionifmatch.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.892701 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="If-Match constraints based on revision_number" alias="revision-if-match" description="Extension indicating that If-Match based on revision_number is supported." updated="2016-12-11T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.893747 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: revision-if-match May 25 15:54:23.894062 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: revisions.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.895194 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Resource revision numbers" alias="standard-attr-revisions" description="This extension will display the revision number of neutron resources." updated="2016-04-11T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.895701 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: standard-attr-revisions May 25 15:54:23.896019 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: router_availability_zone.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.896678 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Router Availability Zone" alias="router_availability_zone" description="Availability zone support for router." updated="2015-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.897201 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: router_availability_zone May 25 15:54:23.897502 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: routerservicetype.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.898403 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Router Service Type" alias="router-service-type" description="Provides router service type" updated="2013-01-29T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.898953 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension router-service-type not supported by any of loaded plugins May 25 15:54:23.899286 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: security_groups_normalized_cidr.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.900164 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Normalized CIDR field for security group rules" alias="security-groups-normalized-cidr" description="Add new field with normalized remote_ip_prefix cidr in SG rule" updated="2020-07-28T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.900669 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: security-groups-normalized-cidr May 25 15:54:23.900988 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: security_groups_port_filtering.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.901884 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Port filtering on security groups" alias="port-security-groups-filtering" description="Provides security groups filtering when listing ports" updated="2018-01-09T09:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.902385 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: port-security-groups-filtering May 25 15:54:23.902702 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: security_groups_remote_address_group.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.903632 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Remote address group id field for security group rules" alias="security-groups-remote-address-group" description="Add new field of remote address group id in SG rules" updated="2020-08-25T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.904135 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: security-groups-remote-address-group May 25 15:54:23.904455 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: securitygroup.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.913834 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="security-group" alias="security-group" description="The security groups extension." updated="2012-10-05T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.914365 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: security-group May 25 15:54:23.914690 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: segment.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.915502 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Segment" alias="segment" description="Segments extension." updated="2016-02-24T17:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.916007 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension segment not supported by any of loaded plugins May 25 15:54:23.916331 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: segments_peer_subnet_host_routes.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.917252 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Segments peer-subnet host routes" alias="segments-peer-subnet-host-routes" description="Add host routes to subnets on a routed network (segments)" updated="2018-06-12T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.917748 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension segments-peer-subnet-host-routes not supported by any of loaded plugins May 25 15:54:23.918067 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: servicetype.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.919182 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Neutron Service Type Management" alias="service-type" description="API for retrieving service providers for Neutron advanced services" updated="2013-01-20T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.919693 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: service-type May 25 15:54:23.920022 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: sorting.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.920975 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Sorting support" alias="sorting" description="Extension that indicates that sorting is enabled." updated="2016-06-12T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.921301 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: sorting May 25 15:54:23.921616 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: standard_attr_segment.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.922511 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Standard Attribute Segment Extension" alias="standard-attr-segment" description="Add standard attributes to Segment resource" updated="2018-04-09T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.923054 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension standard-attr-segment not supported by any of loaded plugins May 25 15:54:23.923374 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: standardattrdescription.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.924005 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="standard-attr-description" alias="standard-attr-description" description="Extension to add descriptions to standard attributes" updated="2016-02-10T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.924526 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: standard-attr-description May 25 15:54:23.924844 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: stateful_security_group.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.925749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Stateful security group" alias="stateful-security-group" description="Indicates if the security group is stateful or not" updated="2019-11-26T09:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.926253 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: stateful-security-group May 25 15:54:23.926568 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: stdattrs_common.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.927165 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Did not find expected name "Stdattrs_common" in /opt/stack/neutron/neutron/extensions/stdattrs_common.py May 25 15:54:23.927477 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnet_dns_publish_fixed_ip.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.928442 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet DNS publish fixed IP" alias="subnet-dns-publish-fixed-ip" description="Support choosing to publish DNS records for IPs from a subnet" updated="2019-05-24T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.928940 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins May 25 15:54:23.929257 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnet_onboard.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.930353 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet Onboard" alias="subnet_onboard" description="Provides support for onboarding subnets into subnet pools" updated="2018-12-18T09:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.930856 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: subnet_onboard May 25 15:54:23.931200 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnet_segmentid_writable.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.932116 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet SegmentID (writable)" alias="subnet-segmentid-writable" description="Provides a writable segment_id attribute for a subnet resource." updated="2018-03-12T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.932613 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension subnet-segmentid-writable not supported by any of loaded plugins May 25 15:54:23.932932 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnet_service_types.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.933833 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet service types" alias="subnet-service-types" description="Provides ability to set the subnet service_types field" updated="2016-03-15T18:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.934342 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: subnet-service-types May 25 15:54:23.934662 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnetallocation.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.935763 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet Allocation" alias="subnet_allocation" description="Enables allocation of subnets from a subnet pool" updated="2015-03-30T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.936265 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: subnet_allocation May 25 15:54:23.936584 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: subnetpool_prefix_ops.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.937233 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Subnet Pool Prefix Operations" alias="subnetpool-prefix-ops" description="Provides support for adjusting the prefix list of subnet pools" updated="2019-02-08T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.937731 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: subnetpool-prefix-ops May 25 15:54:23.938047 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: tag_ports_during_bulk_creation.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.938974 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Tag Ports During Bulk Creation" alias="tag-ports-during-bulk-creation" description="Allow to tag ports during bulk creation" updated="2019-12-29T19:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.939525 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins May 25 15:54:23.939791 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: tagging.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.941354 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Tag support for resources with standard attribute: port, subnet, subnetpool, network, security_group, router, floatingip, policy, trunk, network_segment_range" alias="standard-attr-tag" description="Enables to set tag on resources with standard attribute." updated="2017-01-01T00:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.941873 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: standard-attr-tag May 25 15:54:23.942196 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: timestamp.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.943415 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Resource timestamps" alias="standard-attr-timestamp" description="Adds created_at and updated_at fields to all Neutron resources that have Neutron standard attributes." updated="2016-09-12T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.943923 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loaded extension: standard-attr-timestamp May 25 15:54:23.944243 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: trunk.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.945266 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Trunk Extension" alias="trunk" description="Provides support for trunk ports" updated="2016-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.945779 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension trunk not supported by any of loaded plugins May 25 15:54:23.946098 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: trunk_details.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.947027 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Trunk port details" alias="trunk-details" description="Expose trunk port details" updated="2016-01-01T10:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.947533 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension trunk-details not supported by any of loaded plugins May 25 15:54:23.947851 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: uplink_status_propagation.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.948751 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Uplink status propagation extension" alias="uplink-status-propagation" description="Provides ability to propagate port uplink status." updated="2018-05-31T18:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.949259 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension uplink-status-propagation not supported by any of loaded plugins May 25 15:54:23.949572 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Loading extension file: vlantransparent.py {{(pid=99435) _load_all_extensions_from_path /opt/stack/neutron/neutron/api/extensions.py:445}} May 25 15:54:23.950234 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Ext name="Vlantransparent" alias="vlan-transparent" description="Provides Vlan Transparent Networks" updated="2015-03-23T09:00:00-00:00" {{(pid=99435) _check_extension /opt/stack/neutron/neutron/api/extensions.py:411}} May 25 15:54:23.950730 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extension vlan-transparent not supported by any of loaded plugins May 25 15:54:23.953219 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:network May 25 15:54:23.953946 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnet May 25 15:54:23.956462 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnetpool May 25 15:54:23.957088 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:port May 25 15:54:23.958368 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.958804 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service FLAVORS assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.959466 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.959897 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.960224 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:router May 25 15:54:23.960875 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:floatingip May 25 15:54:23.961574 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service METERING assigned prefix: /metering {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.963030 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_create 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.963397 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_update 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.963744 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Subscribe: > rbac-policy before_delete 55550000 {{(pid=99435) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 15:54:23.964884 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of CountableResource for resource:rbac_policy May 25 15:54:23.965326 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:security_group May 25 15:54:23.965998 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:security_group_rule May 25 15:54:23.966730 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service CORE assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.967088 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnetpool May 25 15:54:23.967414 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] subnetpool is already registered May 25 15:54:23.968059 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service CORE assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.968339 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnetpool May 25 15:54:23.968663 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] subnetpool is already registered May 25 15:54:23.971278 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.971707 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service FLAVORS assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.972304 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.972719 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service L3_ROUTER_NAT assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.973046 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:router May 25 15:54:23.973373 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] router is already registered May 25 15:54:23.973966 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:floatingip May 25 15:54:23.974292 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] floatingip is already registered May 25 15:54:23.974940 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service METERING assigned prefix: /metering {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.975620 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of CountableResource for resource:rbac_policy May 25 15:54:23.975944 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] rbac_policy is already registered May 25 15:54:23.976484 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:security_group May 25 15:54:23.976806 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] security_group is already registered May 25 15:54:23.977392 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:security_group_rule May 25 15:54:23.977797 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] security_group_rule is already registered May 25 15:54:23.978410 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service CORE assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.978732 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnetpool May 25 15:54:23.979085 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] subnetpool is already registered May 25 15:54:23.979691 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.v2.resource_helper [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Service CORE assigned prefix: {{(pid=99435) build_resource_info /opt/stack/neutron/neutron/api/v2/resource_helper.py:77}} May 25 15:54:23.980012 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Creating instance of TrackedResource for resource:subnetpool May 25 15:54:23.980337 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING neutron.quota.resource_registry [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] subnetpool is already registered May 25 15:54:23.980968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: extensions {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.982458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: address-groups {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.984110 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: address-scopes {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.985327 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: agents {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.986471 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: auto-allocated-topology {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.987908 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: availability_zones {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.989160 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: dhcp-networks {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.990566 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: dhcp-agents {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.992269 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: routers {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.994096 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: flavors {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.995278 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: service_profiles {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.996487 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: next_providers {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.997997 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: service_profiles {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:23.999469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: floatingip-pools {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.000663 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: routers {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.003144 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: floatingips {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.004327 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: l3-routers {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.005714 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: l3-agents {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.007146 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: metering-labels {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.008485 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: metering-label-rules {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.009762 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: network-ip-availabilities {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.011028 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: bindings {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.012687 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: quotas {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.014090 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: quotas {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.015529 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: rbac-policies {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.016802 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: security-groups {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.018003 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: security-group-rules {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.019905 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: service-providers {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.021113 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: subnetpools {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.023515 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: subnetpools {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.024960 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.026688 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.141116 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.142913 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.144729 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.146607 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.148315 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.150212 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.151984 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.153879 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron.api.extensions [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Extended resource: tags {{(pid=99435) __init__ /opt/stack/neutron/neutron/api/extensions.py:157}} May 25 15:54:24.159801 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING keystonemiddleware.auth_token [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. May 25 15:54:24.164580 dv-cinder-786384-6-iscsi neutron-server[99435]: WARNING keystonemiddleware.auth_token [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint May 25 15:54:24.173140 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Acquired lock "singleton_lock" {{(pid=99435) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:266}} May 25 15:54:24.173621 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_concurrency.lockutils [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Releasing lock "singleton_lock" {{(pid=99435) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:282}} May 25 15:54:24.174791 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Starting 2 workers May 25 15:54:24.182465 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Started child 99544 {{(pid=99435) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 25 15:54:24.183976 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Started child 99545 {{(pid=99435) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 25 15:54:24.184936 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.185589 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO neutron.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Neutron service started, listening on 0.0.0.0:9696 May 25 15:54:24.186193 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Notify callbacks [] for process, before_spawn {{(pid=99435) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.186387 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [-] (99544) wsgi starting up on http://0.0.0.0:9696 May 25 15:54:24.188551 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.189000 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Starting 1 workers May 25 15:54:24.189876 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [-] (99545) wsgi starting up on http://0.0.0.0:9696 May 25 15:54:24.194267 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Started child 99546 {{(pid=99435) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 25 15:54:24.194778 dv-cinder-786384-6-iscsi neutron-server[99435]: INFO oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Starting 1 workers May 25 15:54:24.197776 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] calling RpcWorker wait() {{(pid=99546) _wait /opt/stack/neutron/neutron/service.py:128}} May 25 15:54:24.198349 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] returning from RpcWorker wait() {{(pid=99546) _wait /opt/stack/neutron/neutron/service.py:135}} May 25 15:54:24.199258 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Started child 99547 {{(pid=99435) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:576}} May 25 15:54:24.199537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.199985 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [-] neutron.plugins.ml2.plugin.Ml2Plugin method start_rpc_listeners called with arguments () {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:24.200088 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG neutron_lib.callbacks.manager [None req-c441c94a-7139-4593-9c21-0ae8f85e0010 None None] Notify callbacks [] for process, after_spawn {{(pid=99435) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.202016 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=99435) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 25 15:54:24.202217 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 15:54:24.202407 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 15:54:24.202620 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 15:54:24.202984 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 15:54:24.203152 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 15:54:24.203663 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.203961 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.204238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.204494 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.204795 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.205346 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [-] Notify callbacks [] for process, after_init {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:24.205920 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.205920 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.205920 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.206114 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.206295 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:24.206384 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.206539 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.206709 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.206977 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.207252 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.207554 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.207841 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.208173 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.208386 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.208562 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.208818 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_source = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.209085 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.209309 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.209581 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] debug = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.209827 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.210319 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.210561 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.210799 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.211041 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.211334 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.211580 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.211865 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.212101 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.212329 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.212543 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.212786 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.213031 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.213248 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.213469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.213729 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.213970 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.214458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] host = dv-cinder-786384-6-iscsi {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.214725 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.215056 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.215255 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.215505 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.215750 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.215972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.216225 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.216469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.216715 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.216956 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.217263 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.217452 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.217661 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_dir = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.217903 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.218063 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_options = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.218270 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.218557 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.218902 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219054 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219193 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219329 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219577 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219761 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.219985 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.220202 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.220417 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.220650 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.220885 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.221126 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.221406 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.221765 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.221919 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.222803 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.222803 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.222803 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.222803 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.223912 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.223912 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.223912 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.223912 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.223912 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.224693 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.224693 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.224693 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.225101 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.225101 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.225363 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.225505 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.225790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.226191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.226346 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.226546 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.227895 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.228673 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.228673 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_journal = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.228673 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_json = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.228891 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.229048 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.229224 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.229470 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.229711 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.229925 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.230142 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.230378 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.230823 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.231130 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.231434 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.231775 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.232024 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.232264 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.232642 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.232762 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.232944 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.233194 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.233487 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.233712 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.234012 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.234251 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.234535 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.234827 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.235113 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.235404 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.235623 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.235885 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.236125 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.236378 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.236721 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.237496 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.237750 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.82/identity {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.238014 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.238250 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.238473 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.238692 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.238947 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.239185 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.239402 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.239756 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.239925 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.240140 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.240361 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.240595 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.240830 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.241060 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.241285 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.241551 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.242068 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.242286 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.242517 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.242801 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.248383 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.249749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.251894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253183 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253183 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253183 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253183 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253568 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.253774 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.254042 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.254320 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.254582 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.254830 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.255121 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.255394 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.255625 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.255880 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.256159 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.256434 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.256684 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.257156 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.257156 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.257891 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.257891 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.258616 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.258616 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.258616 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.258616 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.259196 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.259196 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.259582 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.259787 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.260008 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.260319 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.260581 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.260860 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.261093 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.261428 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.261646 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.261936 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.262220 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.262685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.262685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.263002 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.263794 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.263794 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.263794 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.264158 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.264158 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.264511 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.264614 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.264840 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.265094 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.265312 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.265639 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.265897 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.266310 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.266442 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.266656 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.82/identity {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.270564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272051 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272540 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.272845 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.273041 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.273218 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.273447 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.273691 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.273934 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.274192 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.274417 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.274634 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.274909 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.275137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.275424 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.275680 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.275916 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.276150 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.276418 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.276658 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.276874 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.277141 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.277357 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.277569 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.277785 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.278158 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.278550 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.278960 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.279215 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.279442 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.279702 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.279948 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.280284 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.280549 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.280785 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.281151 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.281377 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.281624 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.281868 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.282113 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.282333 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.282573 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.282968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.283485 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.283770 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.284025 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.284251 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.284469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.284744 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.285003 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.285221 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.285514 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.285764 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.286022 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.286266 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.286503 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.286739 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.286980 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.287223 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.287467 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.287683 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.287932 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.288271 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.288364 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.288585 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.288797 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.289030 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.289246 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.289470 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.289691 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.289926 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.290158 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.290373 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.290587 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.290807 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.291047 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.291371 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.291633 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.291866 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.292165 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.292419 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.292572 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38252) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:24.292684 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.292908 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.293249 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.293469 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.293731 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.293980 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.294203 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.294511 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.294816 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.295155 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.295453 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.295733 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.296010 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.296290 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.296584 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:900:949'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.296865 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.297084 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.297394 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.297732 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.298017 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.298272 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.298519 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.298789 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.299040 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.299262 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.300011 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.300252 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.300475 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.300737 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.301081 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.301351 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.301537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.301790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.302710 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.302710 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.302710 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.303170 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [-] 10.222.0.82 "GET / HTTP/1.1" status: 200 len: 253 time: 0.0070336 May 25 15:54:24.303309 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.303309 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.303309 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 15:54:24.303704 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] Full set of CONF: {{(pid=99435) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:650}} May 25 15:54:24.303848 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 15:54:24.304073 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] Configuration options gathered from: {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 15:54:24.304250 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 15:54:24.305292 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 15:54:24.305437 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ================================================================================ {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 15:54:24.305654 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] agent_down_time = 75 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.305821 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_automatic_dhcp_failover = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.305984 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_automatic_l3agent_failover = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306224 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_bulk = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306368 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allow_overlapping_ips = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306514 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] allowed_conntrack_helpers = [{'amanda': 'tcp'}, {'ftp': 'tcp'}, {'h323': 'udp'}, {'h323': 'tcp'}, {'irc': 'tcp'}, {'netbios-ns': 'udp'}, {'pptp': 'tcp'}, {'sane': 'tcp'}, {'sip': 'udp'}, {'sip': 'tcp'}, {'snmp': 'udp'}, {'tftp': 'udp'}] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306653 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_extensions_path = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306798 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_paste_config = api-paste.ini {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.306968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] api_workers = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307099 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] auth_strategy = keystone {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307266 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backdoor_port = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307427 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backdoor_socket = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307588 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] backlog = 4096 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] base_mac = fa:16:3e:00:00:00 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.307910 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] bind_host = 0.0.0.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308071 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] bind_port = 9696 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308232 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] client_socket_timeout = 900 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308394 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_dir = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308598 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_file = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308731 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] config_source = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.308865 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] control_exchange = neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.309026 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] core_plugin = ml2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.309187 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] debug = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.309334 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] default_availability_zones = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.309524 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.309745 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_agent_notification = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.310005 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_agents_per_network = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.310900 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_lease_duration = 86400 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.310900 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dhcp_load_type = networks {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.310900 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dns_domain = openstacklocal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.310900 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] dvr_base_mac = fa:16:3f:00:00:00 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_dvr = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_new_agents = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_services_on_agents_with_admin_state_down = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_snat_by_default = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] enable_traditional_dhcp = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] external_dns_driver = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] filter_validation = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] global_physnet_mtu = 1500 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.314855 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] graceful_shutdown_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] host = dv-cinder-786384-6-iscsi {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] host_dvr_for_dhcp = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] http_retries = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] instance_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ipam_driver = internal {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ipv6_pd_enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316013 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_net_cidr = 169.254.192.0/18 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_network_physical_name = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] l3_ha_network_type = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_config_append = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_dir = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_options = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316458 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotate_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotate_interval_type = days {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] log_rotation_type = none {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.316929 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] logging_user_identity_format = %(user)s %(tenant)s %(domain)s %(user_domain)s %(project_domain)s {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_allowed_address_pair = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_dns_nameservers = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_header_line = 16384 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_l3_agents_per_router = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317296 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_logfile_count = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317582 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_logfile_size_mb = 200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317582 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_routes = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317740 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] max_subnet_host_routes = 20 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.317901 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_auto_schedule = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.318238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_link_prefix = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.318545 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] network_scheduler_driver = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.318849 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] notify_nova_on_port_data_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.319192 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] notify_nova_on_port_status_changes = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.319371 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] pagination_max_limit = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.319514 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] periodic_fuzzy_delay = 5 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.320508 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] periodic_interval = 40 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.320823 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] publish_errors = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.321128 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_burst = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.321448 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_except_level = CRITICAL {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.321622 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rate_limit_interval = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.321792 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] retry_until_window = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.321947 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_auto_schedule = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.322108 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_distributed = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.322270 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] router_scheduler_driver = neutron.scheduler.l3_agent_scheduler.LeastRoutersScheduler {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.322437 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_response_max_timeout = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.322731 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_response_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.322838 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_state_report_workers = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.323807 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] rpc_workers = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.323972 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] send_events_interval = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.324321 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] service_plugins = ['router', 'neutron.services.metering.metering_plugin.MeteringPlugin'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.325176 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] setproctitle = on {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.325341 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] state_path = /opt/stack/data/neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.325683 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] syslog_log_facility = LOG_USER {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.325911 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] tcp_keepidle = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326083 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_eventlog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_journal = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_json = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.326966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_stderr = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327376 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] use_syslog = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327376 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] vlan_transparent = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327376 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] watch_log_file = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327560 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_default_pool_size = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327637 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_keep_alive = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327771 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:54:24.327959 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_concurrency.disable_process_locking = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.328126 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_concurrency.lock_path = /opt/stack/data/neutron/lock {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.328338 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.connection_string = messaging:// {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.328464 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.328617 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_doc_type = notification {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329005 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_scroll_size = 10000 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329005 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.es_scroll_time = 2m {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329150 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.filter_error_trace = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329365 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.hmac_keys = SECRET_KEY {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329454 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.sentinel_service_name = mymaster {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329618 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.socket_timeout = 0.1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329784 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] profiler.trace_sqlalchemy = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.329966 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.enforce_new_defaults = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.330130 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.enforce_scope = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.330294 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_default_rule = default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.330460 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.330653 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.policy_file = /etc/neutron/policy.json {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.330901 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.331031 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.331181 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.331341 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.331660 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.331845 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332007 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_type = password {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332187 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.auth_url = http://10.222.0.82/identity {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332387 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.cafile = /opt/stack/data/ca-bundle.pem {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332523 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332636 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332795 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.default_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.332960 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.default_domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333124 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333293 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333439 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333602 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333767 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.333932 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.live_migration_events = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.334094 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.334316 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.334406 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.334563 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.334727 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.project_name = service {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.335705 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.region_name = RegionOne {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.335901 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336090 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.system_scope = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336240 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336503 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.trust_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336627 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336732 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.336882 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.user_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.337046 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] nova.username = nova {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.338772 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.338772 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.339129 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.339129 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.339323 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.339445 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.endpoint_type = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] placement.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.340381 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341167 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.auth_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341167 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341167 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341167 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.collect_timing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341167 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.connect_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341743 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.connect_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341743 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.enable_notifications = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.341743 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.endpoint_override = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342079 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342079 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.interface = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342079 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342413 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.max_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342413 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.min_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342642 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342769 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.service_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.342938 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.service_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343064 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.split_loggers = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343186 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.status_code_retries = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343306 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.status_code_retry_delay = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343455 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343619 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.valid_interfaces = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343782 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ironic.version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.343956 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] service_providers.service_provider = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344301 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344464 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344627 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.344954 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345117 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345279 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345441 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345604 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345766 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.345930 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346097 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346297 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346459 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346622 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346785 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.346987 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347130 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347297 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347459 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347622 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347785 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.347948 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348110 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348274 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_rabbit.ssl_version = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348447 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.driver = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348609 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.retry = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348775 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.348939 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_messaging_notifications.transport_url = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349109 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349284 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.default_quota = -1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349447 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_driver = neutron.db.quota.driver.DbQuotaDriver {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349610 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_floatingip = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349773 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_network = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.349940 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_port = 500 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350104 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_rbac_policy = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350267 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_router = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350429 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350592 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_security_group_rule = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350755 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.quota_subnet = 100 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.350923 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] QUOTAS.track_quota_usage = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351102 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] oslo_middleware.enable_proxy_headers_parsing = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351309 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.backends = ['disable_by_file'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351432 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.detailed = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351567 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.disable_by_file_path = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351727 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] healthcheck.path = /healthcheck {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.351914 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352077 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_tenant_name = admin {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352239 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_token = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352402 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.admin_user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352564 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_admin_prefix = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352726 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.352890 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_port = 35357 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.353108 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_protocol = https {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.353290 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_section = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.353435 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_type = password {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354365 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354365 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_url = http://10.222.0.82/identity {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354644 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.auth_version = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354644 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.cache = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354925 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.354925 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355191 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.default_domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.delay_auth_decision = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355537 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355853 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.domain_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.355853 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356082 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.http_connect_timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356202 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356322 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.identity_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356446 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.include_service_catalog = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356816 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.insecure = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.356816 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.interface = public {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357025 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357025 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357253 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357387 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357511 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357646 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.357809 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_secret_key = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.358494 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_security_strategy = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.358494 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcache_use_advanced_pool = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.358494 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.358494 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.password = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.project_name = service {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359619 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.region_name = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359619 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359619 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_token_roles_required = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359901 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.service_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.359901 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.system_scope = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360099 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.token_cache_time = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360099 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.trust_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360302 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360422 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_domain_name = Default {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360583 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.user_id = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360726 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.username = neutron {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.360888 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] keystone_authtoken.www_authenticate_uri = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.361369 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.backend = dogpile.cache.null {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.361369 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.backend_argument = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.361369 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.config_prefix = cache.oslo {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.361717 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.debug_cache_backend = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.361717 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.362463 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.expiration_time = 600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.362594 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_dead_retry = 300 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.362715 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.362835 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_maxsize = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_pool_unused_timeout = 60 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_servers = ['localhost:11211'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.memcache_socket_timeout = 1.0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.proxies = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.364676 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_allowed_ciphers = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.365559 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_cafile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.365559 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_certfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.365559 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_enabled = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.365559 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] cache.tls_keyfile = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.365559 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.capabilities = [21, 12, 1, 2, 19] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.366211 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.366211 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.366211 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.366211 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.366211 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.capabilities = [21, 12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367148 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367148 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367148 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367148 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_dhcp_release.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367641 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367641 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.367641 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368034 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368034 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_ovs_vsctl.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368034 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.capabilities = [21] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368532 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368532 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.368532 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_namespace.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.capabilities = [12] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369137 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.helper_command = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.thread_pool_size = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] privsep_conntrack.user = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.root_helper = sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.369685 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.root_helper_daemon = sudo /usr/local/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370199 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] AGENT.use_helper_for_ns_read = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370199 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_ipset = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370199 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.enable_security_group = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370572 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.firewall_driver = openvswitch {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370572 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] SECURITYGROUP.permitted_ethertypes = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370834 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.extension_drivers = ['port_security'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.370834 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.external_network_type = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.mechanism_drivers = ['openvswitch', 'hyperv'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.overlay_ip_version = 4 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.path_mtu = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372238 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.physical_network_mtus = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.tenant_network_types = ['vlan'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2.type_drivers = ['local', 'flat', 'vlan', 'gre', 'vxlan', 'geneve'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.372790 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_flat.flat_networks = ['physnet1'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373173 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_geneve.max_header_size = 30 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373173 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_geneve.vni_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373173 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_gre.tunnel_id_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373545 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vlan.network_vlan_ranges = ['physnet1:900:949'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373545 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vxlan.vni_ranges = ['1:1000'] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373798 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ml2_type_vxlan.vxlan_group = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.373798 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] OVS_DRIVER.vnic_type_prohibit_list = [] {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374052 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.backend = sqlalchemy {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374329 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374329 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_debug = 0 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374615 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_parameters = {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374615 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_recycle_time = 3600 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.connection_trace = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.374968 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_inc_retry_interval = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375232 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_max_retries = 20 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375232 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_max_retry_interval = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375484 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.db_retry_interval = 1 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375484 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_overflow = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_pool_size = 50 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.375749 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.max_retries = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376030 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.mysql_enable_ndb = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376227 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.mysql_sql_mode = TRADITIONAL {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376227 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.pool_timeout = None {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376495 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.retry_interval = 10 {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376495 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.slave_connection = **** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376761 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.sqlite_synchronous = True {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.376894 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] database.use_db_reconnect = False {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:54:24.377036 dv-cinder-786384-6-iscsi neutron-server[99435]: DEBUG oslo_service.service [-] ******************************************************************************** {{(pid=99435) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 15:54:24.381169 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.services.l3_router.l3_router_plugin [-] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method start_rpc_listeners called with arguments () {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:35.661128 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-603651bf-6498-43bb-adcb-bcde54ef18cf None None] get_active_networks_info from dv-cinder-786384-6-iscsi {{(pid=99546) get_active_networks_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:158}} May 25 15:54:35.740144 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-603651bf-6498-43bb-adcb-bcde54ef18cf None None] No non-hosted networks {{(pid=99546) auto_schedule_networks /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:60}} May 25 15:54:35.747365 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.agentschedulers_db [None req-603651bf-6498-43bb-adcb-bcde54ef18cf None None] DHCP Agent not found on host dv-cinder-786384-6-iscsi {{(pid=99546) list_active_networks_on_active_dhcp_agent /opt/stack/neutron/neutron/db/agentschedulers_db.py:455}} May 25 15:54:35.873191 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9705c49c-69e6-4590-9aa9-b4ccee112c74 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:38.125630 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:38.126282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.126623 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.127057 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.127393 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.127723 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.128051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.128376 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.128868 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.129330 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.130012 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.130594 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.131323 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:38.139071 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServersRpcApi method report_agent_resource_versions called with arguments (, 'Open vSwitch agent', 'dv-cinder-786384-6-iscsi', {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:38.142040 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] neutron.api.rpc.handlers.resources_rpc.ResourcesPushToServerRpcCallback method report_agent_resource_versions called with arguments (,) {'agent_type': 'Open vSwitch agent', 'agent_host': 'dv-cinder-786384-6-iscsi', 'version_map': {'QosPolicy': '1.8', 'Trunk': '1.1', 'SubPort': '1.0', 'Port': '1.7', 'Subnet': '1.1', 'Network': '1.1', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'Log': '1.0', 'PortForwarding': '1.2', 'ConntrackHelper': '1.0', 'AddressGroup': '1.2'}} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:54:38.417943 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ca398b0f-6cd5-42b7-8a2d-cd445fcf8739 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:38.440330 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-cdc78efe-4af4-4b63-8845-c121ee824d4a None None] Retry wrapper got retriable exception: Failed to create a duplicate Agent: for attribute(s) ['agents.uniq_agents0agent_type0host'] with value(s) L3 agent-dv-cinder-786384-6-iscsi {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 15:54:38.440792 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-cdc78efe-4af4-4b63-8845-c121ee824d4a None None] Performing DB retry for function neutron.db.agents_db.AgentDbMixin.create_or_update_agent {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 15:54:39.027972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-cdc78efe-4af4-4b63-8845-c121ee824d4a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:39.056054 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:41.061858 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:41.448415 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.db.l3_hamode_db [None req-9bd1f309-eac0-44db-b6e5-782796cc9df9 None None] Number of active agents lower than max_l3_agents_per_router. L3 agents available: 1 May 25 15:54:41.538261 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9bd1f309-eac0-44db-b6e5-782796cc9df9 None None] Routers set() bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 15:54:41.538666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9bd1f309-eac0-44db-b6e5-782796cc9df9 None None] Router IDs set() for agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:464}} May 25 15:54:42.555553 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 38354) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:43.164398 dv-cinder-786384-6-iscsi neutron-server[99544]: WARNING oslo_policy.policy [None req-61cc38c6-4db3-4063-adc2-69423bcd48b5 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:43.164909 dv-cinder-786384-6-iscsi neutron-server[99544]: WARNING oslo_policy.policy [None req-61cc38c6-4db3-4063-adc2-69423bcd48b5 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:43.382957 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [None req-61cc38c6-4db3-4063-adc2-69423bcd48b5 admin admin] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99544) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:54:43.384182 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-61cc38c6-4db3-4063-adc2-69423bcd48b5 admin admin] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.8263695 May 25 15:54:45.835988 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 38368) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:46.273190 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Request body: {'subnetpool': {'is_default': True, 'prefixes': ['172.20.0.0/16'], 'shared': True, 'default_prefixlen': 26, 'name': 'shared-default-subnetpool-v4'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:54:46.373085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.policy [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=99544) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 25 15:54:46.417968 dv-cinder-786384-6-iscsi neutron-server[99544]: WARNING oslo_policy.policy [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:46.418343 dv-cinder-786384-6-iscsi neutron-server[99544]: WARNING oslo_policy.policy [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:46.427235 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.quota [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Loaded quota_driver: . May 25 15:54:46.442539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:54:46.580908 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Notify callbacks ['neutron.objects.securitygroup.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747116093759', 'neutron.objects.address_group.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747116094166', 'neutron.objects.network.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747116084226', 'neutron.objects.qos.policy.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747116078715', 'neutron.objects.subnetpool.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747115149846', 'neutron.objects.address_scope.RbacNeutronDbObjectMixin.validate_rbac_policy_change-8747115144479', 'neutron.plugins.ml2.plugin.External_net_db_mixin._process_ext_policy_create-16171918', 'neutron.plugins.ml2.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-616460', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-16328759', 'neutron.services.network_ip_availability.plugin.NeutronDbPluginV2.validate_network_rbac_policy_change-16273387'] for rbac-policy, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:46.799360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] Notify callbacks [] for subnetpool, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:46.800462 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-7201de64-dd97-4448-9c11-d6cf6e718bfc admin admin] 10.222.0.82 "POST /v2.0/subnetpools HTTP/1.1" status: 201 len: 739 time: 0.9634829 May 25 15:54:49.482266 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38382) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:50.243245 dv-cinder-786384-6-iscsi neutron-server[99545]: WARNING oslo_policy.policy [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:50.244872 dv-cinder-786384-6-iscsi neutron-server[99545]: WARNING oslo_policy.policy [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:50.467054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Request body: {'network': {'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'name': 'private'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:54:50.468711 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.quota [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Loaded quota_driver: . May 25 15:54:50.485589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:54:50.493246 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Usage tracker for resource:network and tenant:c42fb092cb7c47b190a9f3a947682f2f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:54:50.499484 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:54:50.508977 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:54:50.656119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Made reservation on behalf of c42fb092cb7c47b190a9f3a947682f2f for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:54:50.734697 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.policy [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Loaded default policies from ['neutron'] under neutron.policies entry points {{(pid=99545) register_rules /opt/stack/neutron/neutron/policy.py:73}} May 25 15:54:50.782369 dv-cinder-786384-6-iscsi neutron-server[99545]: WARNING oslo_policy.policy [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:50.782781 dv-cinder-786384-6-iscsi neutron-server[99545]: WARNING oslo_policy.policy [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html. May 25 15:54:50.793662 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:50.815038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:50.831323 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:54:50.835200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Usage tracker for resource:security_group_rule and tenant:c42fb092cb7c47b190a9f3a947682f2f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:54:50.843118 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:54:50.849839 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:54:50.961361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.044598 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.050495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:51.072800 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] vlan segment allocate from pool success with {'vlan_id': 909, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 15:54:51.099154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8747111927525', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.099845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Unsubscribe: segment precommit_create {{(pid=99545) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 25 15:54:51.100345 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Added segment cd843071-47c2-4aaa-ab3a-767e1cf37804 of type vlan for network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e May 25 15:54:51.109556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ac7bad9e-463d-41e4-a098-8c6cb161cd10,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:54:51.169164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.169549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.169921 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.170277 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.170629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.171098 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.171445 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.171789 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.172152 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.172553 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.173018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.173952 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Tracked resource versions report: May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: pushed versions: May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: { 'AddressGroup': {'1.2'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'ConntrackHelper': {'1.0'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Log': {'1.0'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Network': {'1.1'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Port': {'1.7'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'PortForwarding': {'1.2'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'QosPolicy': {'1.8'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroup': {'1.4'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroupRule': {'1.2'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SubPort': {'1.0'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Subnet': {'1.1'}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Trunk': {'1.1'}} May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: consumer versions: May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'ConntrackHelper': '1.0', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Log': '1.0', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Network': '1.1', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Port': '1.7', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'PortForwarding': '1.2', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'QosPolicy': '1.8', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroup': '1.4', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroupRule': '1.2', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SubPort': '1.0', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Subnet': '1.1', May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Trunk': '1.1'}} May 25 15:54:51.175267 dv-cinder-786384-6-iscsi neutron-server[99545]: {{(pid=99545) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 15:54:51.231630 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.181s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:51.238935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.342514 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.344115 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:54:51.344850 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:51.348785 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:51.459134 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Pushing event updated for resources: {'Network': ['ID=450ada1f-ed44-4f7a-a060-e75e1f1ef41e,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:54:51.462323 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:51.472396 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 15:54:51.547907 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 849 time: 2.0644565 May 25 15:54:54.482633 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38404) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:54.984297 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-db30bcc8-e6e3-45df-8dc6-2e1c1ee21f58 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:54:54.985636 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-db30bcc8-e6e3-45df-8dc6-2e1c1ee21f58 admin admin] 10.222.0.82 "GET /v2.0/networks/450ada1f-ed44-4f7a-a060-e75e1f1ef41e HTTP/1.1" status: 200 len: 825 time: 0.5019338 May 25 15:54:55.020415 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2790ff9b-e1a0-412d-9732-322b8ceefaae admin admin] 10.222.0.82 "GET /v2.0/subnetpools/63de684e-37cb-4995-8ab7-8f863b5b6f37 HTTP/1.1" status: 200 len: 734 time: 0.0287511 May 25 15:54:55.029535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Request body: {'subnet': {'ip_version': 4, 'name': 'private-subnet', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'gateway_ip': '172.20.0.1'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:54:55.035065 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:54:55.038972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Usage tracker for resource:subnet and tenant:c42fb092cb7c47b190a9f3a947682f2f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:54:55.041874 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:54:55.051555 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:54:55.169999 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Made reservation on behalf of c42fb092cb7c47b190a9f3a947682f2f for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:54:55.172619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Notify callbacks [] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:55.784419 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:55.788790 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:55.871579 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Pushing event updated for resources: {'Subnet': ['ID=95d121f9-3db7-4b00-ad43-4c91e8231962,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:54:55.877917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:55.878700 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Checking segment: {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:54:55.879092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Network hostable DHCP agents. Network: 450ada1f-ed44-4f7a-a060-e75e1f1ef41e, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 15:54:55.975651 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 15:54:56.027141 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 450ada1f-ed44-4f7a-a060-e75e1f1ef41e) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:54:56.032305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:54:56.033553 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:56.039545 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4ee76c48-c1ac-40f4-9124-42e1ccb2a5af None None] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:54:56.102207 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 854 time: 1.0764120 May 25 15:54:56.422920 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 15:54:56.425359 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:56.655972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Allocated IP 172.20.0.2 (450ada1f-ed44-4f7a-a060-e75e1f1ef41e/95d121f9-3db7-4b00-ad43-4c91e8231962/45737252-cabd-4b0e-9112-7e2b78004f74) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 15:54:56.960915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:56.976663 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Transition to ACTIVE for port object 45737252-cabd-4b0e-9112-7e2b78004f74 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:54:57.002956 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Transition to ACTIVE for port object 45737252-cabd-4b0e-9112-7e2b78004f74 will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:54:57.073207 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:57.078441 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:57.145520 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 15:54:57.148610 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:54:57.154163 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Pushing event updated for resources: {'Port': ['ID=45737252-cabd-4b0e-9112-7e2b78004f74,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:54:57.156604 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 15:54:57.191013 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.191325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.191645 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.191963 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.192277 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.192591 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.192893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.193204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.193524 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.193834 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.194154 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.194424 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Tracked resource versions report: May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: pushed versions: May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: { 'AddressGroup': {'1.2'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'ConntrackHelper': {'1.0'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Log': {'1.0'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Network': {'1.1'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Port': {'1.7'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'PortForwarding': {'1.2'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'QosPolicy': {'1.8'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroup': {'1.4'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroupRule': {'1.2'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SubPort': {'1.0'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Subnet': {'1.1'}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Trunk': {'1.1'}} May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: consumer versions: May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'ConntrackHelper': '1.0', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Log': '1.0', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Network': '1.1', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Port': '1.7', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'PortForwarding': '1.2', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'QosPolicy': '1.8', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroup': '1.4', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroupRule': '1.2', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SubPort': '1.0', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Subnet': '1.1', May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Trunk': '1.1'}} May 25 15:54:57.195605 dv-cinder-786384-6-iscsi neutron-server[99546]: {{(pid=99546) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 15:54:57.235541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:57.268928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Attempting to bind port 45737252-cabd-4b0e-9112-7e2b78004f74 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 15:54:57.269950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Attempting to bind port 45737252-cabd-4b0e-9112-7e2b78004f74 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 15:54:57.269950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Attempting to bind port 45737252-cabd-4b0e-9112-7e2b78004f74 on network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 15:54:57.276541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 15, 54, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 15:54:57.276939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Checking segment: {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:54:57.277411 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:57.277925 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Bound using segment: {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 15:54:57.278460 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Bound port: 45737252-cabd-4b0e-9112-7e2b78004f74, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 15:54:57.278837 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:57.385124 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] For port 45737252-cabd-4b0e-9112-7e2b78004f74, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 15:54:57.389410 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] For port 45737252-cabd-4b0e-9112-7e2b78004f74, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=45737252-cabd-4b0e-9112-7e2b78004f74,segment=NetworkSegment(cd843071-47c2-4aaa-ab3a-767e1cf37804),segment_id=cd843071-47c2-4aaa-ab3a-767e1cf37804)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 15:54:57.456573 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:54:57.460792 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:54:57.556777 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Pushing event updated for resources: {'Port': ['ID=45737252-cabd-4b0e-9112-7e2b78004f74,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:54:57.561807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:54:58.756781 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38414) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:54:59.220626 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [None req-a6482bb4-c2f2-4c76-9676-a80be2cc9f94 admin admin] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:54:59.221798 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a6482bb4-c2f2-4c76-9676-a80be2cc9f94 admin admin] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4638462 May 25 15:55:01.780018 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 38424) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:02.352002 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [None req-a1031efa-c66b-47de-9c11-b0693ca2853c admin admin] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99544) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:55:02.354067 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a1031efa-c66b-47de-9c11-b0693ca2853c admin admin] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.5729146 May 25 15:55:03.698114 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6de1d90f-dff8-4220-b2e3-73b8565b5734 None None] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:55:03.881677 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-17c99534-69a1-4a0b-8875-edc120a635b1 None None] Provisioning for port 45737252-cabd-4b0e-9112-7e2b78004f74 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:55:04.140884 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 45737252-cabd-4b0e-9112-7e2b78004f74 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:04.310945 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 45737252-cabd-4b0e-9112-7e2b78004f74 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:55:04.325750 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 45737252-cabd-4b0e-9112-7e2b78004f74 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:04.325750 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:04.379978 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-11a3d6b9-ce9c-490b-a46d-76654e5adf7b None None] Provisioning complete for port 45737252-cabd-4b0e-9112-7e2b78004f74 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:04.380431 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-11a3d6b9-ce9c-490b-a46d-76654e5adf7b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:04.601929 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:04.629017 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-11a3d6b9-ce9c-490b-a46d-76654e5adf7b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:04.674772 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 45737252-cabd-4b0e-9112-7e2b78004f74, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=45737252-cabd-4b0e-9112-7e2b78004f74,segment=NetworkSegment(cd843071-47c2-4aaa-ab3a-767e1cf37804),segment_id=cd843071-47c2-4aaa-ab3a-767e1cf37804)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 15:55:04.767144 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:04.847751 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:04.977463 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-11a3d6b9-ce9c-490b-a46d-76654e5adf7b None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 15:55:04.978087 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-11a3d6b9-ce9c-490b-a46d-76654e5adf7b None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 15:55:05.001450 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=45737252-cabd-4b0e-9112-7e2b78004f74,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:05.005857 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.160s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:05.552164 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 38440) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:05.661086 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-dd8e6674-1ed8-49f2-8000-02e706d33c97 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.020484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Request body: {'router': {'admin_state_up': True, 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:55:06.026150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:55:06.032996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Usage tracker for resource:router and tenant:c42fb092cb7c47b190a9f3a947682f2f is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:55:06.038234 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:55:06.048162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:55:06.066115 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 45737252-cabd-4b0e-9112-7e2b78004f74 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:06.147093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.149293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.149842 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 45737252-cabd-4b0e-9112-7e2b78004f74 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:06.150209 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.182042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.400233 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.535622 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:06.537668 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-323c1edf-e2d9-4a36-940b-9ad5edd57008 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 709 time: 0.9833550 May 25 15:55:08.084376 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:08.392780 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-15c42de9-dd0d-4864-b808-1526c9cf9c0c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:09.046944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38450) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:09.072456 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:09.499315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [None req-91e91563-867a-4840-a41b-0f79ac06dc99 admin admin] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:55:09.501159 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-91e91563-867a-4840-a41b-0f79ac06dc99 admin admin] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4521492 May 25 15:55:10.986950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.099381 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38470) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:12.273239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Request body: {'network': {'provider:physical_network': 'physnet1', 'admin_state_up': True, 'router:external': True, 'provider:network_type': 'flat', 'is_default': True, 'name': 'public'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:55:12.278698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:55:12.282838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Usage tracker for resource:network and tenant:a0a3a879f6f64dafb18763e645b05d64 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:55:12.286054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:55:12.292270 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:55:12.386583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Made reservation on behalf of a0a3a879f6f64dafb18763e645b05d64 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:55:12.388986 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.403206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.425498 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:55:12.429747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Usage tracker for resource:security_group_rule and tenant:a0a3a879f6f64dafb18763e645b05d64 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:55:12.432908 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:55:12.439286 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:55:12.530021 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.626365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.629690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:12.645945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_flat [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Reserving flat network on physical network physnet1 {{(pid=99545) reserve_provider_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_flat.py:100}} May 25 15:55:12.668828 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c1488172-f8d5-489c-9fd2-64a023246af1,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:12.673678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:12.678642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.678642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Added segment 1a95fb4a-3292-44a5-abc6-7330179a94c6 of type flat for network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc May 25 15:55:12.710420 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.837589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.838948 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:55:12.839530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:12.843007 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:12.945013 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Pushing event updated for resources: {'Network': ['ID=0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:12.947691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:12.956433 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 15:55:12.999113 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.8984559 May 25 15:55:15.388378 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38490) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:15.506847 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-1addaf3d-8020-45f5-9974-cd264d4fb1a8 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:55:15.508263 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1addaf3d-8020-45f5-9974-cd264d4fb1a8 admin admin] 10.222.0.82 "GET /v2.0/subnets/95d121f9-3db7-4b00-ad43-4c91e8231962 HTTP/1.1" status: 200 len: 849 time: 0.1189332 May 25 15:55:15.715294 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8503f51d-bfc3-4a74-b330-0162cba96703 admin admin] 10.222.0.82 "GET /v2.0/routers/cf28515d-072a-4a33-8d3f-2da3fb050a71 HTTP/1.1" status: 200 len: 704 time: 0.2009025 May 25 15:55:16.228189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:16.506216 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Allocated IP 172.20.0.1 (450ada1f-ed44-4f7a-a060-e75e1f1ef41e/95d121f9-3db7-4b00-ad43-4c91e8231962/bcd42729-2e45-461c-a29c-48c3f2024584) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 15:55:16.670546 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:16.690561 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Transition to ACTIVE for port object bcd42729-2e45-461c-a29c-48c3f2024584 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:55:16.895363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:16.900479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:16.960841 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 15:55:16.963817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:55:16.966435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 15:55:16.967151 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.162451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Pushing event updated for resources: {'Port': ['ID=bcd42729-2e45-461c-a29c-48c3f2024584,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:17.166113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.265s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:17.278393 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a1c67e62-b9f4-46b9-9ea7-d81d7c41beaf None None] Provisioning for port bcd42729-2e45-461c-a29c-48c3f2024584 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:55:17.286565 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a1c67e62-b9f4-46b9-9ea7-d81d7c41beaf None None] Provisioning complete for port bcd42729-2e45-461c-a29c-48c3f2024584 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:17.287146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a1c67e62-b9f4-46b9-9ea7-d81d7c41beaf None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.353484 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-a1c67e62-b9f4-46b9-9ea7-d81d7c41beaf None None] Port bcd42729-2e45-461c-a29c-48c3f2024584 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 15:55:17.404644 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.662450 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.723368 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.724787 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:17.728415 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:17.857023 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Pushing event updated for resources: {'Port': ['ID=bcd42729-2e45-461c-a29c-48c3f2024584,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:17.861828 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.133s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:17.925196 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 15:55:17.926472 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:55:18.182197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-473e3297-6aac-44b1-bb10-f149d8c59ef0 None None] Provisioning complete for port bcd42729-2e45-461c-a29c-48c3f2024584 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:18.182597 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-473e3297-6aac-44b1-bb10-f149d8c59ef0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:18.204353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Router cf28515d-072a-4a33-8d3f-2da3fb050a71 is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99545) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 15:55:18.252756 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-473e3297-6aac-44b1-bb10-f149d8c59ef0 None None] Port bcd42729-2e45-461c-a29c-48c3f2024584 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 15:55:18.371803 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 15:55:18.382230 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] 10.222.0.82 "PUT /v2.0/routers/cf28515d-072a-4a33-8d3f-2da3fb050a71/add_router_interface HTTP/1.1" status: 200 len: 508 time: 2.6600895 May 25 15:55:18.384939 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Sync routers for ids ['cf28515d-072a-4a33-8d3f-2da3fb050a71'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 15:55:18.405082 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Routers {'cf28515d-072a-4a33-8d3f-2da3fb050a71'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 15:55:18.405998 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T15:55:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['cf28515d-072a-4a33-8d3f-2da3fb050a71'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:18.606968 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Set router cf28515d-072a-4a33-8d3f-2da3fb050a71 gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 15:55:19.010490 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:17Z', 'revision_number': 2, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port_host': None, '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:17Z', 'revision_number': 2, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:19.015870 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Checking router: cf28515d-072a-4a33-8d3f-2da3fb050a71 for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 15:55:19.016244 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Updating router cf28515d-072a-4a33-8d3f-2da3fb050a71 port bcd42729-2e45-461c-a29c-48c3f2024584 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 15:55:19.073202 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:19.506279 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:19.522859 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Transition to ACTIVE for port object bcd42729-2e45-461c-a29c-48c3f2024584 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:55:19.620530 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:19.627116 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:19.864080 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Pushing event updated for resources: {'Port': ['ID=bcd42729-2e45-461c-a29c-48c3f2024584,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:19.884286 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.884286 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.884774 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.885462 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.885913 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.886167 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.886597 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.886940 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.887278 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.887598 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.887908 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.888214 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Tracked resource versions report: May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: pushed versions: May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: { 'AddressGroup': {'1.2'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'ConntrackHelper': {'1.0'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Log': {'1.0'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Network': {'1.1'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Port': {'1.7'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'PortForwarding': {'1.2'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'QosPolicy': {'1.8'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroup': {'1.4'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroupRule': {'1.2'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SubPort': {'1.0'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Subnet': {'1.1'}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Trunk': {'1.1'}} May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: consumer versions: May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'ConntrackHelper': '1.0', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Log': '1.0', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Network': '1.1', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Port': '1.7', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'PortForwarding': '1.2', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'QosPolicy': '1.8', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroup': '1.4', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroupRule': '1.2', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SubPort': '1.0', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Subnet': '1.1', May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Trunk': '1.1'}} May 25 15:55:19.889329 dv-cinder-786384-6-iscsi neutron-server[99547]: {{(pid=99547) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 15:55:19.896053 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.269s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:19.898158 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Attempting to bind port bcd42729-2e45-461c-a29c-48c3f2024584 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 15:55:19.898962 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Attempting to bind port bcd42729-2e45-461c-a29c-48c3f2024584 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 15:55:19.899073 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Attempting to bind port bcd42729-2e45-461c-a29c-48c3f2024584 on network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 15:55:19.904534 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 15, 55, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 15:55:19.904907 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Checking segment: {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:55:19.905363 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:19.905747 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Bound using segment: {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 15:55:19.906269 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Bound port: bcd42729-2e45-461c-a29c-48c3f2024584, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'cd843071-47c2-4aaa-ab3a-767e1cf37804', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 909, 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 15:55:19.906675 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:19.997316 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] For port bcd42729-2e45-461c-a29c-48c3f2024584, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 15:55:20.002133 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] For port bcd42729-2e45-461c-a29c-48c3f2024584, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bcd42729-2e45-461c-a29c-48c3f2024584,segment=NetworkSegment(cd843071-47c2-4aaa-ab3a-767e1cf37804),segment_id=cd843071-47c2-4aaa-ab3a-767e1cf37804)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 15:55:20.095651 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:20.100270 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:20.264061 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Routers {'cf28515d-072a-4a33-8d3f-2da3fb050a71'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 15:55:20.264061 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T15:55:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['cf28515d-072a-4a33-8d3f-2da3fb050a71'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:20.339141 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Pushing event updated for resources: {'Port': ['ID=bcd42729-2e45-461c-a29c-48c3f2024584,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:20.399285 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.298s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:20.461837 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] Set router cf28515d-072a-4a33-8d3f-2da3fb050a71 gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 15:55:20.736449 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38502) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:20.921909 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5c8f416e-0950-4056-9efb-90d5f3c22e89 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:55:20.922727 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5c8f416e-0950-4056-9efb-90d5f3c22e89 admin admin] Synchronizing usage tracker for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 15:55:20.951817 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:17Z', 'revision_number': 2, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port_host': None, '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:19Z', 'revision_number': 4, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:20.959372 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0537561a-12d9-471c-aeed-e43e9ce61789 None None] The sync data for ids ['cf28515d-072a-4a33-8d3f-2da3fb050a71'] in dv-cinder-786384-6-iscsi is: [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:17Z', 'revision_number': 2, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port_host': None, '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:19Z', 'revision_number': 4, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 15:55:21.049261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5c8f416e-0950-4056-9efb-90d5f3c22e89 admin admin] Unset dirty status for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 15:55:21.050434 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5c8f416e-0950-4056-9efb-90d5f3c22e89 admin admin] 10.222.0.82 "GET /v2.0/networks/0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc HTTP/1.1" status: 200 len: 842 time: 0.3128245 May 25 15:55:21.060630 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Request body: {'subnet': {'name': 'public-subnet', 'ip_version': 4, 'gateway_ip': '172.27.2.1', 'enable_dhcp': False, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'cidr': '172.27.2.0/24'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:55:21.067741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:55:21.071810 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Usage tracker for resource:subnet and tenant:a0a3a879f6f64dafb18763e645b05d64 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:55:21.074896 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:55:21.081965 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:55:21.171563 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Made reservation on behalf of a0a3a879f6f64dafb18763e645b05d64 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:55:21.176157 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Notify callbacks [] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:21.898419 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.objects.ports [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Time-cost: Fetching 0 ports in 0 security groups {{(pid=99545) get_objects /opt/stack/neutron/neutron/objects/ports.py:443}} May 25 15:55:21.899438 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:21.903101 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:21.980726 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Pushing event updated for resources: {'Subnet': ['ID=9237fe36-57f4-4495-8b6e-e6960468c84c,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:21.985729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:22.001912 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:55:22.002423 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Network hostable DHCP agents. Network: 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 15:55:22.112346 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 15:55:22.250730 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:55:22.256344 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:55:22.257275 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:22.262666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6fc43d4d-487f-48a6-a8fb-3840f86aac4f None None] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:55:22.422783 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 821 time: 1.3652444 May 25 15:55:22.486514 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-536906b8-8215-475a-96b4-b48ece4ea2f7 None None] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:55:24.118392 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bcd42729-2e45-461c-a29c-48c3f2024584 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:24.300612 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port bcd42729-2e45-461c-a29c-48c3f2024584 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:55:24.306409 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bcd42729-2e45-461c-a29c-48c3f2024584 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:24.306796 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:24.446692 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:24.480552 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port bcd42729-2e45-461c-a29c-48c3f2024584, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bcd42729-2e45-461c-a29c-48c3f2024584,segment=NetworkSegment(cd843071-47c2-4aaa-ab3a-767e1cf37804),segment_id=cd843071-47c2-4aaa-ab3a-767e1cf37804)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 15:55:24.570174 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:24.572388 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:24.640713 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=bcd42729-2e45-461c-a29c-48c3f2024584,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:24.646621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:24.783746 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38516) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:24.984587 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ac0bb2b8-deb1-4be2-a3c8-4444c92444cd admin admin] 10.222.0.82 "GET /v2.0/routers/cf28515d-072a-4a33-8d3f-2da3fb050a71 HTTP/1.1" status: 200 len: 710 time: 0.1996951 May 25 15:55:25.115992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b4b8645a-ca71-4e19-8e4f-d1d8e07f716e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:55:25.117365 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b4b8645a-ca71-4e19-8e4f-d1d8e07f716e admin admin] 10.222.0.82 "GET /v2.0/networks/0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc HTTP/1.1" status: 200 len: 886 time: 0.1277232 May 25 15:55:25.126585 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Request body: {'router': {'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:55:25.574566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:25.982918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:26.040285 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:26.074557 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bcd42729-2e45-461c-a29c-48c3f2024584 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:26.146038 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bcd42729-2e45-461c-a29c-48c3f2024584 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:26.146437 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:26.183479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Allocated IP 172.27.2.98 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/efe7be7b-34b1-4b49-a207-a8180367b1c3) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 15:55:26.341796 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:26.443569 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:26.447676 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:26.514374 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 15:55:26.517724 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:55:26.522385 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 15:55:26.584129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Pushing event updated for resources: {'Port': ['ID=efe7be7b-34b1-4b49-a207-a8180367b1c3,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:26.586907 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:26.986025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:27.341419 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:27.345358 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_distributed_port_bindings_after_change-490672', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_snat_interfaces_after_change-490737', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._reconfigure_ha_resources-489061'] for router, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:27.573606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Router cf28515d-072a-4a33-8d3f-2da3fb050a71 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 15:55:27.754540 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 15:55:27.764250 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:27.764488 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Sync routers for ids ['cf28515d-072a-4a33-8d3f-2da3fb050a71'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 15:55:27.765934 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] 10.222.0.82 "PUT /v2.0/routers/cf28515d-072a-4a33-8d3f-2da3fb050a71 HTTP/1.1" status: 200 len: 889 time: 2.6427007 May 25 15:55:27.775641 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Routers {'cf28515d-072a-4a33-8d3f-2da3fb050a71'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 15:55:27.776283 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T15:55:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['cf28515d-072a-4a33-8d3f-2da3fb050a71'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:28.012906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Set router cf28515d-072a-4a33-8d3f-2da3fb050a71 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 15:55:28.378272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98'}], 'enable_snat': True}, 'gw_port_id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:26Z', 'revision_number': 4, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port': {'id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:27:b6:d0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:26Z', 'updated_at': '2021-05-25T15:55:26Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:24Z', 'revision_number': 5, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:28.383674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Checking router: cf28515d-072a-4a33-8d3f-2da3fb050a71 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 15:55:28.383957 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Updating router cf28515d-072a-4a33-8d3f-2da3fb050a71 port efe7be7b-34b1-4b49-a207-a8180367b1c3 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 15:55:28.438814 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:28.810354 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:28.824769 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Transition to ACTIVE for port object efe7be7b-34b1-4b49-a207-a8180367b1c3 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:55:28.923089 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:28.927277 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-18d0af44-9312-4782-919a-523e2f7aa494 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:29.006616 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Pushing event updated for resources: {'Port': ['ID=efe7be7b-34b1-4b49-a207-a8180367b1c3,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:29.010497 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:29.011311 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Attempting to bind port efe7be7b-34b1-4b49-a207-a8180367b1c3 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 15:55:29.011754 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Attempting to bind port efe7be7b-34b1-4b49-a207-a8180367b1c3 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 15:55:29.012010 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Attempting to bind port efe7be7b-34b1-4b49-a207-a8180367b1c3 on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 15:55:29.019193 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 15, 55, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 15:55:29.019571 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:55:29.019961 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:29.020345 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 15:55:29.020820 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Bound port: efe7be7b-34b1-4b49-a207-a8180367b1c3, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 15:55:29.021192 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:29.098086 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] For port efe7be7b-34b1-4b49-a207-a8180367b1c3, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 15:55:29.101150 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] For port efe7be7b-34b1-4b49-a207-a8180367b1c3, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=efe7be7b-34b1-4b49-a207-a8180367b1c3,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 15:55:29.169577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:29.174734 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:29.256829 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Pushing event updated for resources: {'Port': ['ID=efe7be7b-34b1-4b49-a207-a8180367b1c3,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:29.259369 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Routers {'cf28515d-072a-4a33-8d3f-2da3fb050a71'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 15:55:29.260105 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 0, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 0, "log_agent_heartbeats": false, "routers": 0},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T15:55:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['cf28515d-072a-4a33-8d3f-2da3fb050a71'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:29.312509 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.138s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:29.446287 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] Set router cf28515d-072a-4a33-8d3f-2da3fb050a71 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 15:55:29.793993 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98'}], 'enable_snat': True}, 'gw_port_id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:26Z', 'revision_number': 4, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port': {'id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:27:b6:d0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:26Z', 'updated_at': '2021-05-25T15:55:29Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:24Z', 'revision_number': 5, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 15:55:29.798940 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-33b38f20-95dc-49d8-b9c5-c50930500c03 None None] The sync data for ids ['cf28515d-072a-4a33-8d3f-2da3fb050a71'] in dv-cinder-786384-6-iscsi is: [{'id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'name': 'router1', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98'}], 'enable_snat': True}, 'gw_port_id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T15:55:06Z', 'updated_at': '2021-05-25T15:55:26Z', 'revision_number': 4, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'gw_port': {'id': 'efe7be7b-34b1-4b49-a207-a8180367b1c3', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:27:b6:d0', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_gateway', 'standard_attr_id': 21, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.98', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:26Z', 'updated_at': '2021-05-25T15:55:29Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bcd42729-2e45-461c-a29c-48c3f2024584', 'name': '', 'network_id': '450ada1f-ed44-4f7a-a060-e75e1f1ef41e', 'tenant_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'mac_address': 'fa:16:3e:72:4b:d4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'cf28515d-072a-4a33-8d3f-2da3fb050a71', 'device_owner': 'network:router_interface', 'standard_attr_id': 19, 'fixed_ips': [{'subnet_id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'ip_address': '172.20.0.1', 'prefixlen': 26}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T15:55:16Z', 'updated_at': '2021-05-25T15:55:24Z', 'revision_number': 5, 'project_id': 'c42fb092cb7c47b190a9f3a947682f2f', 'subnets': [{'id': '95d121f9-3db7-4b00-ad43-4c91e8231962', 'cidr': '172.20.0.0/26', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': '63de684e-37cb-4995-8ab7-8f863b5b6f37'}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 15:55:30.120965 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device efe7be7b-34b1-4b49-a207-a8180367b1c3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:30.232795 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 38526) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:55:30.306135 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port efe7be7b-34b1-4b49-a207-a8180367b1c3 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:55:30.311815 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port efe7be7b-34b1-4b49-a207-a8180367b1c3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:30.315803 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:30.378275 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2851d4a2-37be-42e6-a3d1-338918b04555 admin admin] 10.222.0.82 "GET /v2.0/ports?device_owner=network%3Arouter_gateway&fields=id&fields=name&fields=mac_address&fields=fixed_ips&fields=status HTTP/1.1" status: 200 len: 410 time: 0.1442783 May 25 15:55:30.479458 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:30.508515 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port efe7be7b-34b1-4b49-a207-a8180367b1c3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=efe7be7b-34b1-4b49-a207-a8180367b1c3,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 15:55:30.631816 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:30.634095 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-031ad1ce-c79c-4784-a235-00228e7c4c4f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:55:30.709655 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=efe7be7b-34b1-4b49-a207-a8180367b1c3,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:55:30.713106 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:55:32.081132 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device efe7be7b-34b1-4b49-a207-a8180367b1c3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:55:32.185581 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port efe7be7b-34b1-4b49-a207-a8180367b1c3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:55:32.185990 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:32.258507 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:55:35.628226 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-24ff43bf-e18f-426a-8f22-93a6e25f0ba9 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:38.094679 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:38.472482 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6494f1b7-adc2-4411-b849-20374bc81300 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:39.032025 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:55:41.058361 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:05.614658 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21544cbe-123d-4298-8e96-228eeb865fba None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:08.094057 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:08.396451 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e5e5e0a7-10ae-45e1-ab1a-f2b9eb38d5bf None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:08.991195 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:09.258967 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:56:11.002328 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:35.617123 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-892a7064-f0fb-4192-8d15-65efc310d2fa None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:38.098821 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:38.401702 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98e24a62-a3ce-49a7-aa84-f2207cb4ca51 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:38.982884 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:41.016781 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:56:46.258077 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:57:05.635415 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-62b1062e-2fe2-4e7b-bba4-53fee5eb1e75 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:08.132224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:08.406841 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-47f37d85-7f12-439d-bce4-ac4b0d346a34 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:09.018312 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:10.274828 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39098) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:10.358649 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.pecan_wsgi.hooks.translation [None req-d9d5bca5-990c-464a-aaa7-cf3a4823fd7b admin admin] GET failed (client error): The resource could not be found. May 25 15:57:10.360009 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d9d5bca5-990c-464a-aaa7-cf3a4823fd7b admin admin] 10.222.0.82 "GET /v2.0/networks/private HTTP/1.1" status: 404 len: 315 time: 0.0842004 May 25 15:57:10.519286 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6fcd9f05-f44a-4aa3-9a43-4cfc5421255e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:10.520862 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6fcd9f05-f44a-4aa3-9a43-4cfc5421255e admin admin] 10.222.0.82 "GET /v2.0/networks?name=private HTTP/1.1" status: 200 len: 872 time: 0.1566570 May 25 15:57:11.037291 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:12.677007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39108) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:12.795509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d636fb7b-a93d-48c9-8bdb-b4b46122cb1d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:12.796868 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d636fb7b-a93d-48c9-8bdb-b4b46122cb1d admin admin] 10.222.0.82 "GET /v2.0/subnets/95d121f9-3db7-4b00-ad43-4c91e8231962 HTTP/1.1" status: 200 len: 849 time: 0.1189156 May 25 15:57:12.806240 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Request body: {'subnet': {'dns_nameservers': ['8.8.8.8', '8.8.4.4']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:57:12.927361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Notify callbacks [] for subnet, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:13.218852 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_subnet_gateway_ip_update-8747111867549'] for subnet, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:13.226147 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:57:13.304236 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:13.304917 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:13.306501 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] 10.222.0.82 "PUT /v2.0/subnets/95d121f9-3db7-4b00-ad43-4c91e8231962 HTTP/1.1" status: 200 len: 868 time: 0.5040536 May 25 15:57:13.322621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d76f1fd2-39d6-4f3d-9716-dafea5aafbfb None None] Network 450ada1f-ed44-4f7a-a060-e75e1f1ef41e requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:57:13.349557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Pushing event updated for resources: {'Subnet': ['ID=95d121f9-3db7-4b00-ad43-4c91e8231962,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:57:13.371709 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.372229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.372727 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.373230 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.373719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.374217 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.374712 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.375228 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.375721 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.376197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.376687 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.377173 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Tracked resource versions report: May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: pushed versions: May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: { 'AddressGroup': {'1.2'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'ConntrackHelper': {'1.0'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Log': {'1.0'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Network': {'1.1'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Port': {'1.7'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'PortForwarding': {'1.2'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'QosPolicy': {'1.8'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroup': {'1.4'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroupRule': {'1.2'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SubPort': {'1.0'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Subnet': {'1.1'}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Trunk': {'1.1'}} May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: consumer versions: May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'ConntrackHelper': '1.0', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Log': '1.0', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Network': '1.1', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Port': '1.7', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'PortForwarding': '1.2', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'QosPolicy': '1.8', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroup': '1.4', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroupRule': '1.2', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SubPort': '1.0', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Subnet': '1.1', May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Trunk': '1.1'}} May 25 15:57:13.379030 dv-cinder-786384-6-iscsi neutron-server[99544]: {{(pid=99544) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 15:57:13.383982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:57:13.854084 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-d1acfd66-b205-4832-9d8b-2f2d42525d14 None None] Provisioning complete for port bcd42729-2e45-461c-a29c-48c3f2024584 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:57:13.854534 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d1acfd66-b205-4832-9d8b-2f2d42525d14 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:13.995340 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-d1acfd66-b205-4832-9d8b-2f2d42525d14 None None] Provisioning complete for port 45737252-cabd-4b0e-9112-7e2b78004f74 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:57:13.995654 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d1acfd66-b205-4832-9d8b-2f2d42525d14 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:15.596838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39122) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:15.780948 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a94d4160-2bf8-4ab2-aae4-a8bd210ed935 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:15.781327 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a94d4160-2bf8-4ab2-aae4-a8bd210ed935 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:15.782897 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a94d4160-2bf8-4ab2-aae4-a8bd210ed935 admin admin] 10.222.0.82 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1851075 May 25 15:57:17.972800 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39132) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:18.154967 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-01f85d00-e9ba-4c97-a8a5-916e88d5018c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:18.155349 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-01f85d00-e9ba-4c97-a8a5-916e88d5018c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:18.156868 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-01f85d00-e9ba-4c97-a8a5-916e88d5018c admin admin] 10.222.0.82 "GET /v2.0/networks HTTP/1.1" status: 200 len: 1549 time: 0.1830790 May 25 15:57:20.289499 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39142) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:20.472185 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-613ac6f4-af64-4068-ba62-dbc39c667038 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:20.473433 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-613ac6f4-af64-4068-ba62-dbc39c667038 admin admin] 10.222.0.82 "GET /v2.0/networks/0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc HTTP/1.1" status: 200 len: 886 time: 0.1828444 May 25 15:57:22.658978 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39152) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:22.848156 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-70d255de-8232-4f33-ba62-00207e40f915 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:22.849445 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-70d255de-8232-4f33-ba62-00207e40f915 admin admin] 10.222.0.82 "GET /v2.0/networks/450ada1f-ed44-4f7a-a060-e75e1f1ef41e HTTP/1.1" status: 200 len: 869 time: 0.1893456 May 25 15:57:23.258161 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:57:25.040928 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39162) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:25.158982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0110803e-b347-45cd-8da7-8f3a6c213f58 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:25.159440 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0110803e-b347-45cd-8da7-8f3a6c213f58 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:25.161015 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-0110803e-b347-45cd-8da7-8f3a6c213f58 admin admin] 10.222.0.82 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1191490 May 25 15:57:27.364583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39172) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:27.475222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6795b582-fc14-41e8-9299-4c7e261676d2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:27.475600 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-6795b582-fc14-41e8-9299-4c7e261676d2 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:27.477134 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6795b582-fc14-41e8-9299-4c7e261676d2 admin admin] 10.222.0.82 "GET /v2.0/subnets HTTP/1.1" status: 200 len: 1479 time: 0.1115148 May 25 15:57:29.727729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39182) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:29.833521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cb05cf24-e174-4dcc-8e06-7b5cec1c7f27 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:29.834076 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-cb05cf24-e174-4dcc-8e06-7b5cec1c7f27 admin admin] Synchronizing usage tracker for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:subnet {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 15:57:29.926429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-cb05cf24-e174-4dcc-8e06-7b5cec1c7f27 admin admin] Unset dirty status for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:subnet {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 15:57:29.928374 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-cb05cf24-e174-4dcc-8e06-7b5cec1c7f27 admin admin] 10.222.0.82 "GET /v2.0/subnets/9237fe36-57f4-4495-8b6e-e6960468c84c HTTP/1.1" status: 200 len: 816 time: 0.1994953 May 25 15:57:32.082342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39192) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:32.197575 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ac6b3106-7501-44a4-a265-44ee0646ca85 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:32.198977 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ac6b3106-7501-44a4-a265-44ee0646ca85 admin admin] 10.222.0.82 "GET /v2.0/subnets/95d121f9-3db7-4b00-ad43-4c91e8231962 HTTP/1.1" status: 200 len: 868 time: 0.1156015 May 25 15:57:35.649030 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a2ae4b75-a2a4-48aa-9caa-453993ffb7c1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:38.123430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:38.426280 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f49c985-a2a2-4656-9d8a-1179c1a439fd None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:39.000615 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:41.035601 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:57:55.877022 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39236) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:55.926541 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [None req-4014aca7-ef23-4e4f-8867-91a968ce8b36 admin admin] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:57:55.927565 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-4014aca7-ef23-4e4f-8867-91a968ce8b36 admin admin] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0495760 May 25 15:57:58.222606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39246) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:57:58.310234 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.pecan_wsgi.hooks.translation [None req-76a5d16f-dcc4-46f7-bc59-7593ed5fc6a3 admin admin] GET failed (client error): The resource could not be found. May 25 15:57:58.311983 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-76a5d16f-dcc4-46f7-bc59-7593ed5fc6a3 admin admin] 10.222.0.82 "GET /v2.0/networks/public HTTP/1.1" status: 404 len: 314 time: 0.0879147 May 25 15:57:58.449408 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-67b568e0-8f61-41b8-aa90-fb216ca05ee4 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:57:58.450817 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-67b568e0-8f61-41b8-aa90-fb216ca05ee4 admin admin] 10.222.0.82 "GET /v2.0/networks?name=public HTTP/1.1" status: 200 len: 889 time: 0.1332502 May 25 15:58:00.256714 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:58:00.667926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39256) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:58:00.724939 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Request body: {'network': {'shared': True, 'name': 'shared', 'admin_state_up': True}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:58:00.730586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:58:00.741222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Attempting to reserve 1 items for resource network. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:58:00.880168 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Made reservation on behalf of a0a3a879f6f64dafb18763e645b05d64 for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:58:00.882583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:00.981550 dv-cinder-786384-6-iscsi neutron-server[99544]: WARNING neutron.services.revisions.revision_plugin [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Could not find related network for resource to bump revision. May 25 15:58:00.996159 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] vlan segment allocate from pool success with {'vlan_id': 926, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 15:58:01.011814 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Notify callbacks ['neutron.services.segments.db._add_segment_host_mapping_for_segment-8747111927525', 'neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:01.012583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Unsubscribe: segment precommit_create {{(pid=99544) unsubscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:83}} May 25 15:58:01.013043 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Added segment bdf1b954-6f11-47d3-84b2-de67cb821027 of type vlan for network 8153c8a7-e464-4057-a931-9802f12fc008 May 25 15:58:01.101211 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:01.223576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:01.224930 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:58:01.225509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:01.229178 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:58:01.332069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Pushing event updated for resources: {'Network': ['ID=8153c8a7-e464-4057-a931-9802f12fc008,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:58:01.335587 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:58:01.343259 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 8153c8a7-e464-4057-a931-9802f12fc008 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 15:58:01.435573 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-fd1fcf27-9b97-4189-a426-099f53d389af admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 847 time: 0.7659602 May 25 15:58:03.619980 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39266) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:58:03.689489 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.pecan_wsgi.hooks.translation [None req-3d074095-77d9-4cff-8aef-ce7679101ab3 admin admin] GET failed (client error): The resource could not be found. May 25 15:58:03.690020 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3d074095-77d9-4cff-8aef-ce7679101ab3 admin admin] Synchronizing usage tracker for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:network {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 15:58:03.777270 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3d074095-77d9-4cff-8aef-ce7679101ab3 admin admin] Unset dirty status for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:network {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 15:58:03.778479 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3d074095-77d9-4cff-8aef-ce7679101ab3 admin admin] 10.222.0.82 "GET /v2.0/networks/shared HTTP/1.1" status: 404 len: 314 time: 0.1575453 May 25 15:58:03.884750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-956c409e-02c9-43ff-a1cf-c2a6fb71ef63 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:58:03.886152 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-956c409e-02c9-43ff-a1cf-c2a6fb71ef63 admin admin] 10.222.0.82 "GET /v2.0/networks?name=shared HTTP/1.1" status: 200 len: 826 time: 0.1034880 May 25 15:58:03.895691 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Request body: {'subnet': {'description': 'shared-subnet', 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008', 'ip_version': 4, 'cidr': '192.168.233.0/24', 'name': 'shared-subnet'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 15:58:03.900921 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:58:03.910162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:58:04.001976 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Made reservation on behalf of a0a3a879f6f64dafb18763e645b05d64 for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 15:58:04.004467 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:04.516186 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:04.520620 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-03926455-c6dc-4290-9b1d-4af638f16b50 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:58:04.592599 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:58:04.596545 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:58:04.603444 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Checking segment: {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:58:04.603797 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Network hostable DHCP agents. Network: 8153c8a7-e464-4057-a931-9802f12fc008, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 15:58:04.679904 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Network 8153c8a7-e464-4057-a931-9802f12fc008 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 15:58:04.749863 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 8153c8a7-e464-4057-a931-9802f12fc008) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:58:04.755051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 15:58:04.755051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:04.759378 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c58f4766-781d-4f60-8e14-923431267f5f None None] Network 8153c8a7-e464-4057-a931-9802f12fc008 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:58:04.840998 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 0.9485693 May 25 15:58:04.987077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-8153c8a7-e464-4057-a931-9802f12fc008', 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008', 'tenant_id': 'a0a3a879f6f64dafb18763e645b05d64', 'fixed_ips': [{'subnet_id': '4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 15:58:04.987927 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.159674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Allocated IP 192.168.233.2 (8153c8a7-e464-4057-a931-9802f12fc008/4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb/63555a7a-1b03-44c0-9359-8c3a792615d3) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 15:58:05.372114 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.389730 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Transition to ACTIVE for port object 63555a7a-1b03-44c0-9359-8c3a792615d3 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:58:05.422201 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Transition to ACTIVE for port object 63555a7a-1b03-44c0-9359-8c3a792615d3 will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 15:58:05.577608 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.581953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:58:05.646708 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Network 8153c8a7-e464-4057-a931-9802f12fc008 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 15:58:05.647955 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 15:58:05.652319 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 15:58:05.660880 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f2a120fd-1c51-4f73-9fa7-ff1ee662e913 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.726430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Pushing event updated for resources: {'Port': ['ID=63555a7a-1b03-44c0-9359-8c3a792615d3,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:58:05.752258 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.752589 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.752918 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.753238 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.753564 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.753882 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.754201 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.754519 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.754849 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.755191 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.755499 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.755803 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:58:05.761425 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:58:05.768926 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Attempting to bind port 63555a7a-1b03-44c0-9359-8c3a792615d3 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 15:58:05.769152 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Attempting to bind port 63555a7a-1b03-44c0-9359-8c3a792615d3 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 15:58:05.769519 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Attempting to bind port 63555a7a-1b03-44c0-9359-8c3a792615d3 on network 8153c8a7-e464-4057-a931-9802f12fc008 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 15:58:05.775673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 15, 57, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 15:58:05.776051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Checking segment: {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 15:58:05.776475 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.776870 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Bound using segment: {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 15:58:05.777383 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Bound port: 63555a7a-1b03-44c0-9359-8c3a792615d3, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 15:58:05.777764 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.873438 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] For port 63555a7a-1b03-44c0-9359-8c3a792615d3, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 15:58:05.876603 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] For port 63555a7a-1b03-44c0-9359-8c3a792615d3, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=63555a7a-1b03-44c0-9359-8c3a792615d3,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 15:58:05.966319 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:05.970396 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:58:06.051204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Pushing event updated for resources: {'Port': ['ID=63555a7a-1b03-44c0-9359-8c3a792615d3,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:58:06.054346 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:58:06.846314 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1740e66f-63ef-4e08-8084-69c115b0fc27 None None] Network 8153c8a7-e464-4057-a931-9802f12fc008 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 15:58:07.070379 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-8fe7b67b-c9c7-43bf-947c-83791d712d5a None None] Provisioning for port 63555a7a-1b03-44c0-9359-8c3a792615d3 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:58:08.168506 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:08.215567 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 63555a7a-1b03-44c0-9359-8c3a792615d3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:58:08.365809 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 63555a7a-1b03-44c0-9359-8c3a792615d3 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 15:58:08.370860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 63555a7a-1b03-44c0-9359-8c3a792615d3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:58:08.371250 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:08.423819 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-75bceb82-0630-4ddc-b713-ca99c4e98dab None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:08.490019 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:08.522453 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 63555a7a-1b03-44c0-9359-8c3a792615d3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=63555a7a-1b03-44c0-9359-8c3a792615d3,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 15:58:08.600213 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:08.602318 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:58:08.677353 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=63555a7a-1b03-44c0-9359-8c3a792615d3,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:58:08.681536 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:58:09.005765 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:10.190339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 63555a7a-1b03-44c0-9359-8c3a792615d3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 15:58:10.295514 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 63555a7a-1b03-44c0-9359-8c3a792615d3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 15:58:10.295935 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:11.085847 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:35.629853 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ba19137c-7966-4a7b-8908-175cd4eaca19 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:37.254803 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:58:38.103842 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:38.404905 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c8b7a6d2-e785-4a8c-a4c9-1bae9f03d792 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:39.046610 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:58:41.086922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:05.651703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-5c47e166-0a4c-4a62-a546-4206e0a1ec57 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:08.160019 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:08.530787 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-1f0df369-3b2a-4a49-a411-95d52ef3cb87 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:09.069959 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:11.067729 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:14.254440 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 15:59:28.988319 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39318) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:59:28.994826 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [None req-c4e4f95b-cdb4-4722-8cdd-3fa48f56f1a2 tempest-verify_tempest_config-312205079 tempest-verify_tempest_config-312205079-project] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 15:59:28.996004 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c4e4f95b-cdb4-4722-8cdd-3fa48f56f1a2 tempest-verify_tempest_config-312205079 tempest-verify_tempest_config-312205079-project] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13635 time: 0.0064967 May 25 15:59:29.612298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39336) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:59:29.737789 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:29.753688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 15:59:29.758380 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Usage tracker for resource:security_group_rule and tenant:462e06a9dfc94099b7208beeb2e1ff02 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 15:59:29.761732 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 15:59:29.767500 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 15:59:29.867468 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:29.970839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:29.978615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:59:30.028343 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=462e06a9dfc94099b7208beeb2e1ff02&name=default HTTP/1.1" status: 200 len: 2975 time: 0.4149461 May 25 15:59:30.029060 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2cfa7c61-253b-43d8-82d1-c513819ce829,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:59:30.031339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39342) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 15:59:30.038841 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.039269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.039607 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.039931 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.040247 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.040597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.041095 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.041422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.041749 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.042064 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.042373 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.042681 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.047548 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:59:30.098411 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:30.127312 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:30.313804 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:30.317933 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 15:59:30.331698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2cfa7c61-253b-43d8-82d1-c513819ce829,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 15:59:30.340365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:30.341517 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2cfa7c61-253b-43d8-82d1-c513819ce829 HTTP/1.1" status: 204 len: 168 time: 0.3090153 May 25 15:59:30.342852 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.343240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.343579 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.343912 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.344340 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.344703 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.345074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.345409 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.345763 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.346207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.346898 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.346898 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 15:59:30.350629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 15:59:35.690234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-1db89a29-874f-449f-b557-369238004bc3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:38.118503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:38.427148 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8f6502d9-bfe9-4a0d-b27a-f77832c1e811 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:39.038508 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:41.044726 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 15:59:51.253265 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:00:05.625757 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6bb0f542-93d3-4ad9-9555-c9ed04106d33 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:08.100336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:08.437092 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-1dc90cfc-3327-4c9c-915f-e486ec97b3b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:09.020595 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:11.054630 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:28.252528 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:00:35.614234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-93a62cbb-6ee2-4ad7-96fc-f346799f3769 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:38.099769 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:38.402377 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6b333d03-1a09-4bf6-919e-219fb4ac4bf8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:39.042212 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:00:41.093634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:05.253037 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 5 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:01:05.647765 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4307373e-c046-4116-b387-929ffdbc72e5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:08.121870 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:08.476377 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f8126fb1-5a4d-4b43-99e3-208704118c33 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:08.618629 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534'] for agent, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:09.024901 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:11.081935 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:23.693905 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39414) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:01:23.814471 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-72646e40-99eb-4cf5-be96-17247ac330d2 admin admin] 10.222.0.82 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.1194510 May 25 16:01:26.014796 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39424) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:01:26.079005 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8d87d901-80fa-4609-b8ef-06d4f8fd3508 admin admin] 10.222.0.82 "GET /v2.0/agents HTTP/1.1" status: 200 len: 4303 time: 0.0629070 May 25 16:01:35.686490 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-13d4654e-1d2c-4d24-9f11-364e0348db10 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:38.121411 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:38.474044 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-878b85dd-c145-46f3-9358-6c2866e2fd1f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:38.645953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:39.069770 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:41.085489 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:01:42.252956 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:02:05.672921 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ca1544d7-47b8-4b88-a6fe-5114a64c6dd2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:08.164164 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:08.434954 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-eec403e0-f595-4676-9fba-3b9854ce9ef8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:08.646320 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:09.045938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:11.097888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:19.253714 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:02:35.633783 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6e9853f3-f1c0-4b39-87ce-732866388b1a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:38.128578 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:38.411474 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3d1581cd-e291-495e-ab5d-d4af53ed5e06 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:38.580315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:39.025376 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:41.088718 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:55.842388 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39602) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:56.092451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-network', 'tenant_id': '690d1dab67d340598496a2b626fceaf2'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:56.098022 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:56.102543 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Usage tracker for resource:network and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:56.105654 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:56.113314 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:56.245457 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Made reservation on behalf of 690d1dab67d340598496a2b626fceaf2 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:56.248596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.262229 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:02:56.270502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.285860 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:56.289752 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:56.292799 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:56.294974 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39632) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:56.299678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:56.397469 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.611277 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.615802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:56.653893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=145a6cbf-04aa-45cf-a11a-036b86aa1c9c,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:56.653893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] vlan segment allocate from pool success with {'vlan_id': 943, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:02:56.661798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.662397 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.662730 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.663147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.663468 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.663786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.664164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.664497 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.664825 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.665186 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.665670 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.665982 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Tracked resource versions report: May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: pushed versions: May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: { 'AddressGroup': {'1.2'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'ConntrackHelper': {'1.0'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Log': {'1.0'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Network': {'1.1'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Port': {'1.7'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'PortForwarding': {'1.2'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'QosPolicy': {'1.8'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroup': {'1.4'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroupRule': {'1.2'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SubPort': {'1.0'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Subnet': {'1.1'}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Trunk': {'1.1'}} May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: consumer versions: May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: HyperV agent@hv-cinder-78638: {}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'ConntrackHelper': '1.0', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Log': '1.0', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Network': '1.1', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Port': '1.7', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'PortForwarding': '1.2', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'QosPolicy': '1.8', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroup': '1.4', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SecurityGroupRule': '1.2', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'SubPort': '1.0', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Subnet': '1.1', May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: 'Trunk': '1.1'}} May 25 16:02:56.667103 dv-cinder-786384-6-iscsi neutron-server[99545]: {{(pid=99545) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 16:02:56.671159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:56.675309 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.675642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Added segment e08db650-c58e-47d7-8f08-2ac947c7f402 of type vlan for network dc1f952c-66ab-4757-a4c9-eef7df57dea8 May 25 16:02:56.709210 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.859279 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.863685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:56.863685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:56.866890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f636bf2-e1e8-458a-b03e-893204fcb7f0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:56.970319 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Pushing event updated for resources: {'Network': ['ID=dc1f952c-66ab-4757-a4c9-eef7df57dea8,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:56.976430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.109s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:56.981987 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:02:57.040320 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 890 time: 1.1966190 May 25 16:02:57.053461 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39648) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:57.063189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Request body: {'network': {'name': 'tempest-ServerRescueNegativeTestJSON-275192402-network', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:57.070213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Request body: {'subnet': {'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-subnet', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:57.076461 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.081213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.085902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Usage tracker for resource:network and tenant:d20a1b70ac3f4086aac2870ee8c08efe is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.089242 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Usage tracker for resource:subnet and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.091934 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.094822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.102585 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.104720 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.202554 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Made reservation on behalf of d20a1b70ac3f4086aac2870ee8c08efe for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:57.205112 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.220501 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.232253 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39660) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:57.235674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.239991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Usage tracker for resource:security_group_rule and tenant:d20a1b70ac3f4086aac2870ee8c08efe is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.245160 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.249318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.269123 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39666) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:57.290724 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Made reservation on behalf of 690d1dab67d340598496a2b626fceaf2 for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:57.292944 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.302801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39672) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:57.419670 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.448857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Request body: {'network': {'name': 'tempest-DeleteServersTestJSON-698500773-network', 'tenant_id': '96b2d1f8431044fca9997bb410be55b8'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:57.453535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.459017 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Usage tracker for resource:network and tenant:96b2d1f8431044fca9997bb410be55b8 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.462911 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.469964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.492877 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeNegativeTest-936010552-network', 'tenant_id': 'ed476c44700b47648958b0d2e020320d'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:57.511105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.517635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Usage tracker for resource:network and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.519522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.525189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.536780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Request body: {'network': {'name': 'tempest-AttachVolumeTestJSON-1265607496-network', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:57.547314 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.559299 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.567432 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Usage tracker for resource:network and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.579834 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.585223 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:57.600396 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] vlan segment allocate from pool success with {'vlan_id': 945, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:02:57.628360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.628973 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Added segment c781c6c6-6d43-4217-aae6-6935286facf2 of type vlan for network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 May 25 16:02:57.640714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c255724c-5dbd-46d2-bb27-077184561f8d,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:57.655761 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Tracked resource versions report: May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: pushed versions: May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: { 'AddressGroup': {'1.2'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'ConntrackHelper': {'1.0'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Log': {'1.0'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Network': {'1.1'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Port': {'1.7'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'PortForwarding': {'1.2'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'QosPolicy': {'1.8'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroup': {'1.4'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroupRule': {'1.2'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SubPort': {'1.0'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Subnet': {'1.1'}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Trunk': {'1.1'}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: consumer versions: May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: HyperV agent@hv-cinder-78638: {}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'ConntrackHelper': '1.0', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Log': '1.0', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Network': '1.1', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Port': '1.7', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'PortForwarding': '1.2', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'QosPolicy': '1.8', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroup': '1.4', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SecurityGroupRule': '1.2', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'SubPort': '1.0', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Subnet': '1.1', May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: 'Trunk': '1.1'}} May 25 16:02:57.663117 dv-cinder-786384-6-iscsi neutron-server[99544]: {{(pid=99544) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 16:02:57.672128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Made reservation on behalf of 96b2d1f8431044fca9997bb410be55b8 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:57.675950 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.691554 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.693736 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.700356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:57.725553 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.734087 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.742649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Usage tracker for resource:security_group_rule and tenant:96b2d1f8431044fca9997bb410be55b8 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.756358 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.764221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.809855 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:57.813122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.845056 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.879364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:57.887019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Usage tracker for resource:security_group_rule and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:57.891634 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:57.900160 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:57.922644 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.925655 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:57.926311 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.958478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Made reservation on behalf of 06dda51337ff4f03a0a16af8eaf39a13 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:57.961132 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:57.961391 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:57.979962 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.018929 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.021197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:58.029919 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:58.033830 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:58.044410 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:58.128396 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.147394 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.149949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.150643 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Pushing event updated for resources: {'Network': ['ID=feb61d72-e6b4-48c6-bc7c-d61f51987cf9,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:58.157525 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.196s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:58.157999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:58.160150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:58.168285 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:02:58.212932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=367321a9-0437-4990-9bd8-2e314882d97c,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:58.218712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:58.237844 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Pushing event updated for resources: {'Subnet': ['ID=21c0b655-f786-4f4e-807c-e0313e0d89ae,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:58.238217 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] vlan segment allocate from pool success with {'vlan_id': 901, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:02:58.241697 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:58.249964 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Checking segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:02:58.250503 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Network hostable DHCP agents. Network: dc1f952c-66ab-4757-a4c9-eef7df57dea8, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:02:58.255834 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.277479 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 891 time: 1.9805946 May 25 16:02:58.289422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39704) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:58.289996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] vlan segment allocate from pool success with {'vlan_id': 948, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:02:58.296787 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Request body: {'subnet': {'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'cidr': '172.20.0.0/28', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-subnet', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:58.305621 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.305820 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Added segment be9421e6-1214-4783-b6f4-e62225b27ecf of type vlan for network 9123c0ac-e64c-41a9-8d68-d98246284959 May 25 16:02:58.306191 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:58.316736 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Usage tracker for resource:subnet and tenant:d20a1b70ac3f4086aac2870ee8c08efe is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:58.324526 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:58.345188 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:58.357498 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-93488be8-69c5-40b1-b281-5a53b07b6dbc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:58.367155 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.372362 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Added segment fcbe700b-26f1-40db-9201-5ec1c3805964 of type vlan for network 5e540876-3bc6-4d60-aed5-9bca9dafe63d May 25 16:02:58.372560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:02:58.375635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.431173 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Made reservation on behalf of d20a1b70ac3f4086aac2870ee8c08efe for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:58.433695 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.489489 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=651bb4e0-fd9c-4049-a285-3f148efba400,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:58.494054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:58.497679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.508549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, dc1f952c-66ab-4757-a4c9-eef7df57dea8) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:02:58.520270 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-40527500-afb0-47f8-9926-554816b01280 None None] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:02:58.532671 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.559642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:58.560238 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.605725 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.615665 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:58.642129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.669835 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:58.673503 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.716423 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] vlan segment allocate from pool success with {'vlan_id': 932, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:02:58.737574 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.739384 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Added segment 4e483d36-603d-4ea7-a03e-3b742a52a6dc of type vlan for network 0105f026-2520-418a-a172-829f4b156413 May 25 16:02:58.767435 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 853 time: 1.7227068 May 25 16:02:58.780769 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.790712 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39718) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:58.806726 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:58.836402 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:58.842007 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.848718 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Usage tracker for resource:router and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:58.855959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:58.874817 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:58.907754 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:58.908599 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.927736 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4dec3eef-5a85-4ac9-9518-ae4455aabf44,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:58.931824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.317s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:58.962248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0dddd49f-d631-42c8-9fb6-7dc2eaa3e9f7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.203s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:58.994356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:58.995817 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.016544 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:02:59.017433 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.070010 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.110847 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.137810 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:02:59.143127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.189994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Pushing event updated for resources: {'Network': ['ID=9123c0ac-e64c-41a9-8d68-d98246284959,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:59.203676 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 9123c0ac-e64c-41a9-8d68-d98246284959 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:02:59.276214 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 1.9718821 May 25 16:02:59.280543 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 2.0065520 May 25 16:02:59.290131 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39736) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:59.290695 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39738) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:59.292496 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 2.0589850 May 25 16:02:59.296089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39740) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:59.301571 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Request body: {'subnet': {'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d', 'cidr': '172.20.0.0/28', 'name': 'tempest-DeleteServersTestJSON-698500773-subnet', 'tenant_id': '96b2d1f8431044fca9997bb410be55b8', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:59.310172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:59.315580 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Pushing event updated for resources: {'Network': ['ID=5e540876-3bc6-4d60-aed5-9bca9dafe63d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:59.317991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Usage tracker for resource:subnet and tenant:96b2d1f8431044fca9997bb410be55b8 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:59.320712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.359s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:59.321117 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.407s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:59.336065 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 5e540876-3bc6-4d60-aed5-9bca9dafe63d details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:02:59.354566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:59.370228 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Request body: {'subnet': {'network_id': '0105f026-2520-418a-a172-829f4b156413', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeTestJSON-1265607496-subnet', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'enable_dhcp': True, 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:59.376002 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Request body: {'subnet': {'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'cidr': '172.20.0.0/28', 'name': 'tempest-AttachVolumeNegativeTest-936010552-subnet', 'tenant_id': 'ed476c44700b47648958b0d2e020320d', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:02:59.390372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:59.411233 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:59.428621 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Usage tracker for resource:subnet and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:59.439659 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:59.448829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Pushing event updated for resources: {'Network': ['ID=0105f026-2520-418a-a172-829f4b156413,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:02:59.450031 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:59.451705 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:59.452110 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.293s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:59.452437 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:02:59.469528 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 0105f026-2520-418a-a172-829f4b156413 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:02:59.496816 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Allocated IP 172.20.0.2 (dc1f952c-66ab-4757-a4c9-eef7df57dea8/21c0b655-f786-4f4e-807c-e0313e0d89ae/3ec2b131-bb9b-40db-b38d-53666e2fa8ee) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:02:59.508515 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Made reservation on behalf of 96b2d1f8431044fca9997bb410be55b8 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:59.511560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.531634 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39758) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:02:59.674471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Usage tracker for resource:subnet and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:59.677416 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:59.679711 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:59.743601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.750620 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:59.768489 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.780084 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Made reservation on behalf of 06dda51337ff4f03a0a16af8eaf39a13 for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:59.782329 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.786304 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:02:59.790288 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Usage tracker for resource:security_group_rule and tenant:e36c0726c9ed4cce9dc1afbeda4080bf is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:02:59.793223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:02:59.799241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:02:59.856431 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.867181 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.873397 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3c812690-1b93-430e-a591-be4c60704f9e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:59.880414 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Transition to ACTIVE for port object 3ec2b131-bb9b-40db-b38d-53666e2fa8ee will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:02:59.904397 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Transition to ACTIVE for port object 3ec2b131-bb9b-40db-b38d-53666e2fa8ee will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:02:59.909309 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:02:59.911521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.912010 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.974274 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.976604 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:02:59.985423 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:02:59.986310 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:00.031493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Synchronizing usage tracker for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:00.036734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f6b7dd36-fed3-49f1-a2f3-7e5b1b30877b,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:00.040901 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:00.093565 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:00.097104 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:00.101080 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:00.153891 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Unset dirty status for tenant:a0a3a879f6f64dafb18763e645b05d64 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:00.159573 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=e36c0726c9ed4cce9dc1afbeda4080bf&name=default HTTP/1.1" status: 200 len: 2975 time: 0.6246197 May 25 16:03:00.164331 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39776) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:00.166009 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Pushing event updated for resources: {'Subnet': ['ID=051377f2-6b21-4071-b8f5-21107e3c870f,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:00.168957 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.296s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:00.201708 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Checking segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:00.202071 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Network hostable DHCP agents. Network: feb61d72-e6b4-48c6-bc7c-d61f51987cf9, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:03:00.210838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Pushing event updated for resources: {'Port': ['ID=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:00.225235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.225835 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.226311 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.226790 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.227274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.227779 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.228096 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.228569 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.228906 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.229224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.229692 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.230139 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.230601 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Tracked resource versions report: May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: pushed versions: May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: { 'AddressGroup': {'1.2'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'ConntrackHelper': {'1.0'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Log': {'1.0'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Network': {'1.1'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Port': {'1.7'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'PortForwarding': {'1.2'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'QosPolicy': {'1.8'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroup': {'1.4'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroupRule': {'1.2'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SubPort': {'1.0'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Subnet': {'1.1'}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Trunk': {'1.1'}} May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: consumer versions: May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: HyperV agent@hv-cinder-78638: {}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'ConntrackHelper': '1.0', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Log': '1.0', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Network': '1.1', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Port': '1.7', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'PortForwarding': '1.2', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'QosPolicy': '1.8', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroup': '1.4', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SecurityGroupRule': '1.2', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'SubPort': '1.0', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Subnet': '1.1', May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: 'Trunk': '1.1'}} May 25 16:03:00.231937 dv-cinder-786384-6-iscsi neutron-server[99546]: {{(pid=99546) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 16:03:00.234734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:00.238368 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.253s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:00.239422 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Attempting to bind port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:00.239956 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Attempting to bind port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:00.240473 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Attempting to bind port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:00.247288 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 2, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:00.247906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Checking segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:00.248467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.249000 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Bound using segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:00.249626 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Bound port: 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:00.250149 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.349549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Pushing event updated for resources: {'Subnet': ['ID=7a4c9393-23ba-4625-b1b4-895efa8276f1,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:00.349807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] For port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:00.353603 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] For port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:00.355031 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.120s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:00.356938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Checking segment: {'id': 'fcbe700b-26f1-40db-9201-5ec1c3805964', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:00.357590 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Network hostable DHCP agents. Network: 5e540876-3bc6-4d60-aed5-9bca9dafe63d, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:03:00.420537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.424934 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:00.465058 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:03:00.479651 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Network 5e540876-3bc6-4d60-aed5-9bca9dafe63d is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:03:00.504862 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Pushing event updated for resources: {'Port': ['ID=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:00.508001 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:00.512908 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.578963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 5e540876-3bc6-4d60-aed5-9bca9dafe63d) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:00.583566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:00.584568 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.585888 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.590467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0516207d-26e7-4a86-819b-a09b5b53057e None None] Network 5e540876-3bc6-4d60-aed5-9bca9dafe63d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:00.662705 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, feb61d72-e6b4-48c6-bc7c-d61f51987cf9) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:00.667278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:00.667894 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:00.677173 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-23c59516-0825-4dcb-bf89-b7df1f639a44 None None] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:00.729374 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 848 time: 1.4240305 May 25 16:03:00.998224 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 855 time: 2.7045538 May 25 16:03:00.998529 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39802) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:01.006398 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Request body: {'router': {'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:01.014149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:01.025843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Usage tracker for resource:router and tenant:d20a1b70ac3f4086aac2870ee8c08efe is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:01.029932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:01.055239 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.071892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:01.094542 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.127920 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bc81122f-c72f-419b-8e7f-ebe8402116ce None None] Network 5e540876-3bc6-4d60-aed5-9bca9dafe63d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:01.146574 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.178864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:01.193878 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.198981 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.214335 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f6b7dd36-fed3-49f1-a2f3-7e5b1b30877b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:01.222571 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.043s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:01.236579 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-dc44f2aa-f5ce-45ae-a8ee-36a452957c4d None None] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:01.237190 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.305777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.314952 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f6b7dd36-fed3-49f1-a2f3-7e5b1b30877b HTTP/1.1" status: 204 len: 168 time: 1.1446364 May 25 16:03:01.396042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Allocated IP 172.27.2.28 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/9e92c7c2-9cdb-4242-9af8-0873fad5e72a) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:01.752008 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.765533 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:01.780175 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:01.878953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f32f9877-5ada-4b6b-9db3-62bc901308eb None None] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:01.919666 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.085008 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.157318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:02.175162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:02.196719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Checking segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:02.210368 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Network hostable DHCP agents. Network: 9123c0ac-e64c-41a9-8d68-d98246284959, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:03:02.231892 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:02.240933 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-5b7cfbc4-3362-4816-bfb3-7855d993a31f None None] Provisioning for port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:02.272363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.316842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Pushing event updated for resources: {'Subnet': ['ID=b19f7657-6038-4479-b9ec-9e4e92c69bb0,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:02.491551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 3ec2b131-bb9b-40db-b38d-53666e2fa8ee up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:02.549425 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Pushing event updated for resources: {'Subnet': ['ID=c52f7a79-f047-4ca9-9f61-86cb7e470fe5,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:02.569938 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.789s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:02.575832 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.529s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:02.577581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-424efae1-e7e0-4cd1-9fc2-f085c45e7c61 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:02.618242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39892) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:02.663493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.664013 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.515s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:02.676784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:02.685704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Usage tracker for resource:security_group_rule and tenant:c38e3e57bb414e90b07a01bb57c15931 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:02.689122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:02.695809 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:02.716189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Checking segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:02.716633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Network hostable DHCP agents. Network: 0105f026-2520-418a-a172-829f4b156413, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:03:02.733453 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:02.738818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:02.743721 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.877607 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Network 9123c0ac-e64c-41a9-8d68-d98246284959 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:03:02.885607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:02.909491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Pushing event updated for resources: {'Port': ['ID=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:02.922956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.266s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:02.978291 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Network 0105f026-2520-418a-a172-829f4b156413 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:03:02.979444 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-8a653041-b79b-4724-807e-b3aa96ed88a9 None None] Provisioning complete for port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:02.979818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8a653041-b79b-4724-807e-b3aa96ed88a9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.038909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.045194 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.077308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:03.146454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8a653041-b79b-4724-807e-b3aa96ed88a9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.176679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-52175fda-7a36-46ad-a6e4-0bffb0a56c77 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:03.200893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.263994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39912) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:03.272571 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-8a653041-b79b-4724-807e-b3aa96ed88a9 None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 16:03:03.273044 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-8a653041-b79b-4724-807e-b3aa96ed88a9 None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 16:03:03.274774 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:03.286094 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 9123c0ac-e64c-41a9-8d68-d98246284959) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:03.287341 dv-cinder-786384-6-iscsi neutron-server[99545]: WARNING keystonemiddleware.auth_token [-] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 25 16:03:03.287833 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 0105f026-2520-418a-a172-829f4b156413) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:03.299358 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.306171 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=c38e3e57bb414e90b07a01bb57c15931&name=default HTTP/1.1" status: 200 len: 2975 time: 0.6825972 May 25 16:03:03.334131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=99221f96-1f05-4247-b59a-ff78805e9fb6,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:03.357512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39920) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:03.357677 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:03.360826 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-cbda6c5e-31e4-4f2e-86a6-b1516f892bbd None None] Network 9123c0ac-e64c-41a9-8d68-d98246284959 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:03.375621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:03.378323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:03.382074 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.382245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.203s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:03.466290 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:03.466892 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.468694 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.483479 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-bc14ff2f-49ba-4c35-8f76-190e72b63ff1 None None] Network 0105f026-2520-418a-a172-829f4b156413 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:03.499962 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ffce7d7f-962f-4111-9029-227fe2e4ab2d admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 4.1931875 May 25 16:03:03.512885 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.594701 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.603783 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 4.3022513 May 25 16:03:03.604876 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39932) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:03.640180 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:03.644664 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Request body: {'router': {'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:03.693678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:03.712236 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-e7826865-b23e-4c6d-8b7a-6487d94a0e47 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:03.712921 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-e7826865-b23e-4c6d-8b7a-6487d94a0e47 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Synchronizing usage tracker for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:03.804816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Usage tracker for resource:router and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:03.816194 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:03.829150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:03.864708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Allocated IP 172.27.2.194 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/c8688276-a340-4491-aefe-0346cae7a12a) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:03.928336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9a7b6339-2a9c-4106-ba90-ddf19e4c2893 None None] Network 9123c0ac-e64c-41a9-8d68-d98246284959 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:03.992397 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.014522 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.022822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.063508 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-0105f026-2520-418a-a172-829f4b156413', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:03:04.064349 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.145830 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.209100 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.228387 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 39962) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:04.229254 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-e7826865-b23e-4c6d-8b7a-6487d94a0e47 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Unset dirty status for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:04.230604 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-e7826865-b23e-4c6d-8b7a-6487d94a0e47 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/networks?id=5e540876-3bc6-4d60-aed5-9bca9dafe63d HTTP/1.1" status: 200 len: 811 time: 0.9603865 May 25 16:03:04.256630 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.279544 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:04.297710 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=99221f96-1f05-4247-b59a-ff78805e9fb6,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:04.307413 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.312781 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/99221f96-1f05-4247-b59a-ff78805e9fb6 HTTP/1.1" status: 204 len: 168 time: 0.9463701 May 25 16:03:04.324490 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.331098 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:04.331249 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-dda4b5c1-770d-44c8-9aa5-28061c0f0482 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/quotas/96b2d1f8431044fca9997bb410be55b8 HTTP/1.1" status: 200 len: 375 time: 0.0735693 May 25 16:03:04.353907 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:04.395816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:04.406282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:04.408817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:04.462950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Allocated IP 172.20.0.2 (0105f026-2520-418a-a172-829f4b156413/c52f7a79-f047-4ca9-9f61-86cb7e470fe5/9a1700fb-f69b-45cd-a01d-f45daaa969bb) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:04.471601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Router 1145505b-5e57-4698-b277-66ca05d2520b is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99544) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:03:04.492185 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 3ec2b131-bb9b-40db-b38d-53666e2fa8ee up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:04.614069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39972) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:04.742608 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-c9546fe8-acd5-4be2-a700-f98918478048 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=96b2d1f8431044fca9997bb410be55b8&fields=id HTTP/1.1" status: 200 len: 210 time: 0.3928070 May 25 16:03:04.751445 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:04.766146 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:04.787574 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:04.787574 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:02:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:04.864471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.868524 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 928 time: 6.0662816 May 25 16:03:04.869176 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 39984) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:04.880820 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.896743 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:04.896743 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:04.931335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Pushing event updated for resources: {'Port': ['ID=c8688276-a340-4491-aefe-0346cae7a12a,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:04.940422 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.586s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:05.066829 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:05.076866 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Transition to ACTIVE for port object 9a1700fb-f69b-45cd-a01d-f45daaa969bb will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:05.110749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.112699 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.143403 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-ecbb5443-721e-4752-92cd-a09edffecaeb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:05.144383 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-ecbb5443-721e-4752-92cd-a09edffecaeb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:05.159175 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:05.165948 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Usage tracker for resource:security_group_rule and tenant:254005ff7bc043fb876fd05a099f99d2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:05.191092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-d158c95d-f071-4f16-aee7-2f45d6649a7b service neutron] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 16:03:05.198574 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-d158c95d-f071-4f16-aee7-2f45d6649a7b service neutron] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0791655 May 25 16:03:05.214688 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Transition to ACTIVE for port object 9a1700fb-f69b-45cd-a01d-f45daaa969bb will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:05.233536 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:05.256160 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:05.360351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.365474 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:05.408707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-ecbb5443-721e-4752-92cd-a09edffecaeb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:05.409888 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-ecbb5443-721e-4752-92cd-a09edffecaeb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 1.1777914 May 25 16:03:05.474065 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Network 0105f026-2520-418a-a172-829f4b156413 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:05.478263 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:05.482750 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:05.531871 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:02Z', 'revision_number': 3, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:01Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:05.538054 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-00287636-e9bc-4668-94e3-5aeb22ac76d2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/quotas/ed476c44700b47648958b0d2e020320d HTTP/1.1" status: 200 len: 375 time: 0.1118646 May 25 16:03:05.541094 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:05.544037 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Updating router 1145505b-5e57-4698-b277-66ca05d2520b port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:05.563845 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Pushing event updated for resources: {'Port': ['ID=9a1700fb-f69b-45cd-a01d-f45daaa969bb,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:05.572898 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.207s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:05.612981 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.658257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Attempting to bind port 9a1700fb-f69b-45cd-a01d-f45daaa969bb on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:05.658663 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Attempting to bind port 9a1700fb-f69b-45cd-a01d-f45daaa969bb by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:05.659050 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Attempting to bind port 9a1700fb-f69b-45cd-a01d-f45daaa969bb on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:05.666474 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 2, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:05.666853 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Checking segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:05.667688 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.668149 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Bound using segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:05.671446 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Bound port: 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:05.674011 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.734855 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.748568 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-f120a60a-e088-4d3d-a28f-488dbf4e8592 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0736697 May 25 16:03:05.777712 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-729ccdd3-844b-4189-a89b-3bf09fd4f45b service neutron] 10.222.0.82 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5779939 May 25 16:03:05.810380 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e93908a8-c88f-4ff4-94d9-8426c5deeb51 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.813373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.904537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] For port 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:05.908236 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.908924 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] For port 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9a1700fb-f69b-45cd-a01d-f45daaa969bb,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:05.979602 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:05.986150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ececdfd8-5f9b-4afb-946b-29574280c526 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:05.986515 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:06.001377 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=254005ff7bc043fb876fd05a099f99d2&name=default HTTP/1.1" status: 200 len: 2975 time: 1.3862512 May 25 16:03:06.008054 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40016) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:06.058119 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=392ead68-f2fd-467b-9a34-5d6ac18b2eb5,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:06.089970 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:06.120783 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Pushing event updated for resources: {'Port': ['ID=9a1700fb-f69b-45cd-a01d-f45daaa969bb,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:06.138843 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.149s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:06.202488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.236522 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-5aa499dc-fa2e-4692-9cc5-0bc1a42c6dee service neutron] 10.222.0.82 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4505274 May 25 16:03:06.294145 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.299544 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.338169 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Transition to ACTIVE for port object 9e92c7c2-9cdb-4242-9af8-0873fad5e72a will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:06.360275 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.439006 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.446591 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:06.561657 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Attempting to bind port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:06.562129 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Attempting to bind port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:06.562577 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Attempting to bind port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:06.573258 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Pushing event updated for resources: {'Port': ['ID=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,revision_number=2']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 2, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Bound port: 9e92c7c2-9cdb-4242-9af8-0873fad5e72a, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:06.578738 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.587809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.602078 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-d71dc9e4-58a7-4065-bc7a-87bb2bac6f7f service neutron] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 16:03:06.603117 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-d71dc9e4-58a7-4065-bc7a-87bb2bac6f7f service neutron] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.4926803 May 25 16:03:06.646724 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.647999 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.647999 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.648224 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.648617 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.649013 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.649400 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.649788 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.650275 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.650664 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.651106 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.651564 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Tracked resource versions report: May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: pushed versions: May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: { 'AddressGroup': {'1.2'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'ConntrackHelper': {'1.0'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Log': {'1.0'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Network': {'1.1'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Port': {'1.7'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'PortForwarding': {'1.2'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'QosPolicy': {'1.8'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroup': {'1.4'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroupRule': {'1.2'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SubPort': {'1.0'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Subnet': {'1.1'}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Trunk': {'1.1'}} May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: consumer versions: May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: { DHCP agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: HyperV agent@hv-cinder-78638: {}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: L3 agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: Metadata agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: Metering agent@dv-cinder-786384-6-iscsi: {}, May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: Open vSwitch agent@dv-cinder-786384-6-iscsi: { 'AddressGroup': '1.2', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'ConntrackHelper': '1.0', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Log': '1.0', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Network': '1.1', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Port': '1.7', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'PortForwarding': '1.2', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'QosPolicy': '1.8', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroup': '1.4', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SecurityGroupRule': '1.2', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'SubPort': '1.0', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Subnet': '1.1', May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: 'Trunk': '1.1'}} May 25 16:03:06.653003 dv-cinder-786384-6-iscsi neutron-server[99547]: {{(pid=99547) report /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:175}} May 25 16:03:06.665378 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.218s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:06.711367 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] For port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:06.717987 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] For port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:06.756370 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-cc63cc8b-62b8-4d2c-a239-6f100a2e83e2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:06.786713 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=392ead68-f2fd-467b-9a34-5d6ac18b2eb5,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:06.789963 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:06.823902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.825274 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/392ead68-f2fd-467b-9a34-5d6ac18b2eb5 HTTP/1.1" status: 204 len: 168 time: 0.8159621 May 25 16:03:06.886123 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:06.892119 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5dc6edb5-482b-4550-a674-a045e82e6a09 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:06.924026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Allocated IP 172.20.0.1 (dc1f952c-66ab-4757-a4c9-eef7df57dea8/21c0b655-f786-4f4e-807c-e0313e0d89ae/8e6bec27-45dc-4320-bf1a-16e932839558) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:07.041831 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:07.042846 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 1, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 1},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:02:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:07.177699 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Pushing event updated for resources: {'Port': ['ID=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:07.182458 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.290s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:07.238747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.252488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.267417 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1a9d9f60-9572-46e5-8166-fa58fb9627ba req-9485a767-a9a0-42be-8fa4-ddbc9444bfbd tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.3090897 May 25 16:03:07.297802 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Transition to ACTIVE for port object 8e6bec27-45dc-4320-bf1a-16e932839558 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:07.359863 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:07.408687 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.414162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.423999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9bdc5569-7f66-41ad-af7e-15ba6eb9cb12 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:07.563259 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-1f6e181f-d963-4308-b0fa-a5d2ffb1a5ef None None] Network 0105f026-2520-418a-a172-829f4b156413 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:03:07.592397 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.615389 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Pushing event updated for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:07.626114 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.203s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:07.632738 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:07.634666 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:07.653252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:07.653949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:07.816931 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-fd931f4f-f9d4-4862-a631-9b4938ca49b2 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 1.1144061 May 25 16:03:07.970388 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:02Z', 'revision_number': 3, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:06Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:07.977281 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6b1d50ad-15b9-4995-92d2-d20501818edd None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:02Z', 'revision_number': 3, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:06Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:08.089452 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-bc41d21a-5842-43e5-ac49-8c7dbabf20e6 None None] Provisioning for port 9a1700fb-f69b-45cd-a01d-f45daaa969bb completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:08.145951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40078) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:08.284381 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.393984 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-06885984-b772-4b74-8db8-5f6b1433b70b None None] Provisioning for port 8e6bec27-45dc-4320-bf1a-16e932839558 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:08.395642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.407950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-06885984-b772-4b74-8db8-5f6b1433b70b None None] Provisioning complete for port 8e6bec27-45dc-4320-bf1a-16e932839558 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:08.409101 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-06885984-b772-4b74-8db8-5f6b1433b70b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.580002 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-0e37dfe4-681a-4e7a-8f4b-238b3b96c006 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.581887 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5f10001-f868-449f-8e73-7c9a7ea181de req-ba9554d1-a168-4bdc-852c-5f09e6ef6768 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.1680923 May 25 16:03:08.603705 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:08.611737 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Usage tracker for resource:security_group_rule and tenant:644b6e0d58bf4a7a8061a60a2fbd1300 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:08.617997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:08.629149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Allocated IP 172.27.2.71 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/e2560d2e-a52a-4dbc-a74c-eda92a478df8) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:08.629811 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:08.649576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.651448 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-9c3a87bb-53a9-437a-a066-4cf49cba6f54 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.7981129 May 25 16:03:08.713577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9a1700fb-f69b-45cd-a01d-f45daaa969bb up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:08.815202 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-06885984-b772-4b74-8db8-5f6b1433b70b None None] Port 8e6bec27-45dc-4320-bf1a-16e932839558 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:08.826558 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:08.840822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Router d1ce813e-f0ed-493f-8885-b9884ac8f21e is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99545) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:03:08.934834 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 9a1700fb-f69b-45cd-a01d-f45daaa969bb completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:08.953554 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9a1700fb-f69b-45cd-a01d-f45daaa969bb triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:08.967248 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.107772 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-56e44e81-3018-4978-aace-fc4789667015 None None] Provisioning complete for port 9a1700fb-f69b-45cd-a01d-f45daaa969bb triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:09.108401 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-56e44e81-3018-4978-aace-fc4789667015 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.267410 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.305239 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.338047 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.390221 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.408223 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9a1700fb-f69b-45cd-a01d-f45daaa969bb,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:09.409527 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-56e44e81-3018-4978-aace-fc4789667015 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.460015 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:09.470357 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.500365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.512029 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.517508 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b01101f7-8277-4003-a7af-6a8a1567663b req-ee5fb5e8-26ea-4655-a740-745e41b9786a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.0949650 May 25 16:03:09.547137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:09.561063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.568070 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=644b6e0d58bf4a7a8061a60a2fbd1300&name=default HTTP/1.1" status: 200 len: 2975 time: 1.2391787 May 25 16:03:09.568070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=94460178-6fce-4cf7-b5c6-8b0303ee0a4c,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:09.579813 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40122) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:09.583773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.608020 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:09.608535 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:09.744912 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:09.763735 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-36a9f717-aa8a-428c-b172-1114a9bfd4d9 req-819ac546-5464-4d51-a384-f30066fbaeda tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.0725327 May 25 16:03:09.788713 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.836524 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.841148 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4511f823-1b24-4662-b024-db88e450251f None None] Sync routers for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:09.868047 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.873071 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 930 time: 8.8734660 May 25 16:03:09.874162 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40148) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:09.878926 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:09.920372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Pushing event updated for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:09.925202 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:09.926126 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:09.928325 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.320s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:09.980320 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bb2005b-3b50-4f53-bfff-2bff66dd66b0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:10.033312 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:10.035843 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:10.059501 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-56e44e81-3018-4978-aace-fc4789667015 None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 16:03:10.060257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-56e44e81-3018-4978-aace-fc4789667015 None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 16:03:10.073382 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=9a1700fb-f69b-45cd-a01d-f45daaa969bb,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:10.079409 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.337s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:10.114047 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:10.116942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Pushing event updated for resources: {'Port': ['ID=e2560d2e-a52a-4dbc-a74c-eda92a478df8,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:10.129301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:10.154546 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:10.221648 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:10.299786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:10.425465 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:10.469431 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:03:10.819257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9a1700fb-f69b-45cd-a01d-f45daaa969bb up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:10.828894 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:10.890051 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:10.890674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9a1700fb-f69b-45cd-a01d-f45daaa969bb triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:10.890674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:10.890816 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] 10.222.0.82 "PUT /v2.0/routers/1145505b-5e57-4698-b277-66ca05d2520b/add_router_interface HTTP/1.1" status: 200 len: 503 time: 5.9410527 May 25 16:03:10.893674 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:10.893674 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:10.907922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ab12d34f-ad26-4f6b-a559-7877d83da87b None None] Provisioning complete for port 8e6bec27-45dc-4320-bf1a-16e932839558 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:10.908325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab12d34f-ad26-4f6b-a559-7877d83da87b None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:10.952268 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:11.089473 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-ab12d34f-ad26-4f6b-a559-7877d83da87b None None] Port 8e6bec27-45dc-4320-bf1a-16e932839558 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:11.201771 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:05Z', 'revision_number': 3, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:03Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:11.204514 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=94460178-6fce-4cf7-b5c6-8b0303ee0a4c,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:11.211396 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:11.212352 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4511f823-1b24-4662-b024-db88e450251f None None] Checking router: d1ce813e-f0ed-493f-8885-b9884ac8f21e for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:11.212806 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4511f823-1b24-4662-b024-db88e450251f None None] Updating router d1ce813e-f0ed-493f-8885-b9884ac8f21e port c8688276-a340-4491-aefe-0346cae7a12a binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:11.213635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-539af8eb-1774-417f-801e-ca7b86bb6993 req-b95f6def-5a6a-4762-ab47-659726001d13 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.3906307 May 25 16:03:11.223727 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.271s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:11.242467 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:11.243804 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/94460178-6fce-4cf7-b5c6-8b0303ee0a4c HTTP/1.1" status: 204 len: 168 time: 1.6615138 May 25 16:03:11.257522 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c0a8d90-5576-4a0e-b3e7-f51435a018db req-1111bf0e-74e0-47a9-8c12-d37f8b7dd0cf tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.1155086 May 25 16:03:11.312124 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:11.343603 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9e92c7c2-9cdb-4242-9af8-0873fad5e72a up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:11.475327 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:11.895556 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:11.905263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:11.925866 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:11.926249 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:11.966965 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:06Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 2, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:11.972830 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:11.973190 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Updating router 1145505b-5e57-4698-b277-66ca05d2520b port 8e6bec27-45dc-4320-bf1a-16e932839558 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:12.044876 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.169665 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.189155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4511f823-1b24-4662-b024-db88e450251f None None] Transition to ACTIVE for port object c8688276-a340-4491-aefe-0346cae7a12a will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:12.257870 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.291814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.320999 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:12.398047 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Pushing event updated for resources: {'Port': ['ID=c8688276-a340-4491-aefe-0346cae7a12a,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:12.408228 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:12.430063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.449818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:12.454807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-4511f823-1b24-4662-b024-db88e450251f None None] Attempting to bind port c8688276-a340-4491-aefe-0346cae7a12a on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:12.455184 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-4511f823-1b24-4662-b024-db88e450251f None None] Attempting to bind port c8688276-a340-4491-aefe-0346cae7a12a by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:12.455551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4511f823-1b24-4662-b024-db88e450251f None None] Attempting to bind port c8688276-a340-4491-aefe-0346cae7a12a on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:12.461888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4511f823-1b24-4662-b024-db88e450251f None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:12.462278 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4511f823-1b24-4662-b024-db88e450251f None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:12.462690 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.463286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4511f823-1b24-4662-b024-db88e450251f None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:12.463650 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-4511f823-1b24-4662-b024-db88e450251f None None] Bound port: c8688276-a340-4491-aefe-0346cae7a12a, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:12.467134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.484159 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40230) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:12.490158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1686244828', 'description': 'tempest-description--1022919750'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:12.499355 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:12.514604 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:12.520214 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:12.525007 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.536511 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:12.540263 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:12.583067 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.599941 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Transition to ACTIVE for port object 8e6bec27-45dc-4320-bf1a-16e932839558 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:12.633975 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5599c766-6d7a-4425-a2a2-6cc9801287a7 req-4b0c61b3-9175-416b-9475-4f752b1c1d1a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.1855168 May 25 16:03:12.643384 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:12.644787 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4511f823-1b24-4662-b024-db88e450251f None None] For port c8688276-a340-4491-aefe-0346cae7a12a, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:12.650654 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4511f823-1b24-4662-b024-db88e450251f None None] For port c8688276-a340-4491-aefe-0346cae7a12a, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=c8688276-a340-4491-aefe-0346cae7a12a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:12.650809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.650928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:12.679794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:12.683070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.42', 57057) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:12.686831 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:12.699458 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.709610 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f3055933-195f-4df4-b005-5dd5a5a97a58 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:12.770826 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4511f823-1b24-4662-b024-db88e450251f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:12.776497 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:12.866446 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:12.882983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:13.037334 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Pushing event updated for resources: {'Port': ['ID=c8688276-a340-4491-aefe-0346cae7a12a,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:13.058108 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.281s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:13.072839 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Attempting to bind port 8e6bec27-45dc-4320-bf1a-16e932839558 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:13.073343 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Attempting to bind port 8e6bec27-45dc-4320-bf1a-16e932839558 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:13.073473 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Attempting to bind port 8e6bec27-45dc-4320-bf1a-16e932839558 on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:13.084003 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:13.091706 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:13.092825 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:13.093213 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Checking segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:13.097653 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.097653 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Bound using segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:13.097653 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Bound port: 8e6bec27-45dc-4320-bf1a-16e932839558, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:13.097653 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.119062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.185771 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Allocated IP 172.20.0.1 (feb61d72-e6b4-48c6-bc7c-d61f51987cf9/051377f2-6b21-4071-b8f5-21107e3c870f/9ca03cbe-10dd-4e7e-a9ef-5a36837d3215) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:13.237997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.310197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0c43888d-df7d-44c8-addb-f9ec10327c68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:13.357185 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] For port 8e6bec27-45dc-4320-bf1a-16e932839558, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:13.369221 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] For port 8e6bec27-45dc-4320-bf1a-16e932839558, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=8e6bec27-45dc-4320-bf1a-16e932839558,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:13.379492 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Pushing event updated for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:13.392307 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.683s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:13.415145 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.424945 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-a7224e43-c73c-4b4e-8867-dbc92b441e7b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:13.434548 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-a7224e43-c73c-4b4e-8867-dbc92b441e7b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.42 "GET /v2.0/networks?id=5e540876-3bc6-4d60-aed5-9bca9dafe63d HTTP/1.1" status: 200 len: 811 time: 0.7419498 May 25 16:03:13.439814 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.441489 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1780 time: 0.9566557 May 25 16:03:13.447332 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40258) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:13.464747 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.466170 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8279b5d1-27dc-4c22-996b-cb9765a604b0 req-71a35e07-8bd5-4aa9-a324-50950f1b4a9a tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.6976461 May 25 16:03:13.471119 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:13.472258 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group_rule': {'security_group_id': '32ab79bb-f281-465f-93f1-440cbf7dec8c', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:13.485566 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:13.495777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:13.503384 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:13.508086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.511032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:13.526522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroup': ['ID=32ab79bb-f281-465f-93f1-440cbf7dec8c,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:13.530904 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.221s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:13.535864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Request body: {'port': {'device_id': '904361ad-111e-47c6-9c15-338ac12347de', 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d', 'admin_state_up': True, 'tenant_id': '96b2d1f8431044fca9997bb410be55b8'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:13.607492 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:13.608198 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:13.631672 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:13.648986 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.680451 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Pushing event updated for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:13.697407 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.226s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:13.710137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:13.932750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:13.935756 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:13.950150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:13.963318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Usage tracker for resource:port and tenant:96b2d1f8431044fca9997bb410be55b8 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:13.971458 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:13.980218 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:14.031049 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.42', 57064) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:14.032113 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:14.162139 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7070cfb-5af0-4d8e-987c-8f60ef7c62ae req-33f643c7-cdb0-469a-ad0a-5914375595c7 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.2326069 May 25 16:03:14.163093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Pushing event updated for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:14.172799 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:14.193577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:14.194375 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:14.200831 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.493s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:14.243211 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-4511f823-1b24-4662-b024-db88e450251f None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:05Z', 'revision_number': 3, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:14.249357 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4511f823-1b24-4662-b024-db88e450251f None None] The sync data for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi is: [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:05Z', 'revision_number': 3, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:14.286152 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Made reservation on behalf of 96b2d1f8431044fca9997bb410be55b8 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:14.289573 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:14.424268 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99545) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:03:14.486034 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:14.510285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-306a8b01-fd8e-4874-93eb-6ae4bc819ab5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:14.516199 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-306a8b01-fd8e-4874-93eb-6ae4bc819ab5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.4814985 May 25 16:03:14.562479 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Request body: {'port': {'device_id': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'admin_state_up': True, 'tenant_id': 'ed476c44700b47648958b0d2e020320d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:14.630707 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9e92c7c2-9cdb-4242-9af8-0873fad5e72a up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:14.662279 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:13Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:14.667495 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-225f062e-bf2a-49d8-9777-aa3339413d25 None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:13Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:14.844459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:14.950711 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:14.951399 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:15.375799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:15.392068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:15.492658 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Allocated IP 172.20.0.2 (5e540876-3bc6-4d60-aed5-9bca9dafe63d/7a4c9393-23ba-4625-b1b4-895efa8276f1/8c1bac9a-e2d6-43eb-9551-100ddcab0607) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:15.565369 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0e0d32ab-8978-4020-adef-f7d869308b08 req-2e7011bc-c53f-4ebe-9730-0e0e5ed710d2 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.6336284 May 25 16:03:15.639243 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6b49aa5-70c6-4715-9afd-385bdb47fda9 req-25c27d38-9491-437f-9792-af19bc1380a0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.2010391 May 25 16:03:15.741070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=7c93aa6b-64cc-495a-ad1b-bc7f06d71717,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:15.844291 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.468s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:15.848659 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:15.867827 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Usage tracker for resource:port and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:15.874808 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:15.889875 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:15.987718 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:15.997508 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:15.999967 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 2.5485780 May 25 16:03:16.006663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40332) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:16.032959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group_rule': {'security_group_id': '32ab79bb-f281-465f-93f1-440cbf7dec8c', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:16.047193 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:16.061193 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:16.072761 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:16.077459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:16.085577 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:16.112132 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:16.135797 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:16.158148 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:16.176128 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:16.187484 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:16.341121 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:16.348954 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 922 time: 12.7307546 May 25 16:03:16.353110 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40342) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:16.664991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:16.671070 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:16.711887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:16.742323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:16.863641 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:16.928106 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device c8688276-a340-4491-aefe-0346cae7a12a up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:17.084983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Pushing event updated for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:17.117656 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.262s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:17.175444 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Network 5e540876-3bc6-4d60-aed5-9bca9dafe63d is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:17.201223 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:17.256076 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:17.269651 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:17.269651 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.332891 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Allocated IP 172.20.0.11 (9123c0ac-e64c-41a9-8d68-d98246284959/b19f7657-6038-4479-b9ec-9e4e92c69bb0/3935ec55-2a13-43b8-b0bf-f9ef947051dc) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:17.372936 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.468354 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 3.9386334 May 25 16:03:17.515848 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-3e743dca-69fb-483a-9b59-f020c7bb0436 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99544) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 16:03:17.519127 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-3e743dca-69fb-483a-9b59-f020c7bb0436 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.42 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0135324 May 25 16:03:17.544230 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.544644 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.565598 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:11Z', 'revision_number': 3, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:17.579980 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:17.601050 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Updating router b1fcb15a-38af-4be8-95c4-8843c2278f7c port e2560d2e-a52a-4dbc-a74c-eda92a478df8 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:17.601258 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:17.608583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.612717 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7284a855-01c7-460f-b472-456bbd9a6694 req-76860183-4bee-42ea-ba2c-08655f30e009 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.3164594 May 25 16:03:17.638127 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=682e1836-6593-482e-acf0-ff042436c8ae,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:17.650611 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5acca7d5-d11d-47ce-b609-3ffc97a864f6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:17.677240 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port c8688276-a340-4491-aefe-0346cae7a12a completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:17.727133 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port c8688276-a340-4491-aefe-0346cae7a12a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:17.727516 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.792951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40372) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:17.792951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.186s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:17.837068 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:17.985109 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Request body: {'port': {'device_id': '904361ad-111e-47c6-9c15-338ac12347de', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:18.114645 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.116351 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 2.1060734 May 25 16:03:18.134234 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40382) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:18.135398 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.147110 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.156270 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:18.162731 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:18.167195 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:floatingip and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:18.170969 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:18.172128 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port c8688276-a340-4491-aefe-0346cae7a12a, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=c8688276-a340-4491-aefe-0346cae7a12a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:18.179485 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:18.217554 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:18.256998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Usage tracker for resource:security_group_rule and tenant:de1dec2214c148a6a4de8ee83f12008a is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:18.265619 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:18.276979 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:18.279594 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.285961 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.289503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 8e6bec27-45dc-4320-bf1a-16e932839558 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:18.347284 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.374286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:18.375714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.570143 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=c8688276-a340-4491-aefe-0346cae7a12a,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:18.579294 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.205s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:18.584075 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Pushing event updated for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:18.595046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.944s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:18.672937 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.770205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 8e6bec27-45dc-4320-bf1a-16e932839558 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:18.774566 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.793875 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.796612 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 8e6bec27-45dc-4320-bf1a-16e932839558 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:18.797030 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.825340 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:18.839177 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e52cd2b2-be5c-4af2-b469-3efd0ba064c8 req-362d5497-1590-4119-b4ac-7cb268b050f9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:18.846777 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e52cd2b2-be5c-4af2-b469-3efd0ba064c8 req-362d5497-1590-4119-b4ac-7cb268b050f9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 931 time: 1.4675789 May 25 16:03:18.863274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Transition to ACTIVE for port object e2560d2e-a52a-4dbc-a74c-eda92a478df8 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:18.931041 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Network 9123c0ac-e64c-41a9-8d68-d98246284959 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:18.966581 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.981693 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:18.988879 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:19.008081 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:19.030642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:19.050141 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.057949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:19.064484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:19.065217 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.070608 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Pushing event updated for resources: {'Port': ['ID=e2560d2e-a52a-4dbc-a74c-eda92a478df8,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:19.074552 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:19.103960 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 8e6bec27-45dc-4320-bf1a-16e932839558, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=8e6bec27-45dc-4320-bf1a-16e932839558,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:19.127544 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Attempting to bind port e2560d2e-a52a-4dbc-a74c-eda92a478df8 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:19.128063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Attempting to bind port e2560d2e-a52a-4dbc-a74c-eda92a478df8 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:19.135147 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Attempting to bind port e2560d2e-a52a-4dbc-a74c-eda92a478df8 on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:19.135283 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:19.135656 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:19.136094 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.137058 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:19.137649 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Bound port: e2560d2e-a52a-4dbc-a74c-eda92a478df8, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:19.138230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.147608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:19.169159 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:19.214056 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e52cd2b2-be5c-4af2-b469-3efd0ba064c8 req-de4c8375-3030-48d0-903c-c435d63a10b9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Synchronizing usage tracker for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:19.252539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.427s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:19.255379 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=de1dec2214c148a6a4de8ee83f12008a&name=default HTTP/1.1" status: 200 len: 2979 time: 1.4559462 May 25 16:03:19.256395 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-248a217d-3d8c-4d34-9a53-3ec9b09d3650 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.152s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:19.262549 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] For port e2560d2e-a52a-4dbc-a74c-eda92a478df8, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:19.266805 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] For port e2560d2e-a52a-4dbc-a74c-eda92a478df8, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=e2560d2e-a52a-4dbc-a74c-eda92a478df8,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:19.269855 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.272714 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:19.302704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40396) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:19.395088 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device c8688276-a340-4491-aefe-0346cae7a12a up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:19.435344 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6a55392b-bd49-4082-889d-c7f77ed50499,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:19.440117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.183s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:19.455108 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 4.9174674 May 25 16:03:19.474651 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e52cd2b2-be5c-4af2-b469-3efd0ba064c8 req-de4c8375-3030-48d0-903c-c435d63a10b9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Unset dirty status for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:19.475917 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e52cd2b2-be5c-4af2-b469-3efd0ba064c8 req-de4c8375-3030-48d0-903c-c435d63a10b9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.6282980 May 25 16:03:19.489376 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Router d1ce813e-f0ed-493f-8885-b9884ac8f21e has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:03:19.501694 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:19.505719 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e68ae416-740c-4d7d-9bd4-c179581f1c77 req-f82f81ec-f7b7-47f7-a5f4-01e0e615e158 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:19.507434 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e68ae416-740c-4d7d-9bd4-c179581f1c77 req-f82f81ec-f7b7-47f7-a5f4-01e0e615e158 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 932 time: 0.6226792 May 25 16:03:19.519369 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.522510 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.531242 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.258s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:19.532040 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:19.664599 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e68ae416-740c-4d7d-9bd4-c179581f1c77 req-ee27d50f-f9a2-4c27-a22f-f15bba601595 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:19.741520 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:19.747442 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:19.821191 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:19.929594 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Pushing event updated for resources: {'Port': ['ID=e2560d2e-a52a-4dbc-a74c-eda92a478df8,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:19.956010 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.424s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:19.993563 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port c8688276-a340-4491-aefe-0346cae7a12a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:19.994229 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.002502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Allocated IP 172.20.0.1 (0105f026-2520-418a-a172-829f4b156413/c52f7a79-f047-4ca9-9f61-86cb7e470fe5/709f3425-4852-48b5-bffc-a7829c94b719) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:20.192592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.211802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Transition to ACTIVE for port object 709f3425-4852-48b5-bffc-a7829c94b719 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:20.214262 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:20.237256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e68ae416-740c-4d7d-9bd4-c179581f1c77 req-ee27d50f-f9a2-4c27-a22f-f15bba601595 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:20.246913 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e68ae416-740c-4d7d-9bd4-c179581f1c77 req-ee27d50f-f9a2-4c27-a22f-f15bba601595 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.7131772 May 25 16:03:20.258137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 8c1bac9a-e2d6-43eb-9551-100ddcab0607 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:20.282209 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Request body: {'port': {'device_id': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:20.398451 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.411416 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.448126 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:20.450979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.457955 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Sync routers for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:20.477186 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b264f2ec-c7b4-4bb5-bf62-e8517c8bbfeb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:20.478294 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:20.479126 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:20.494814 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.537346 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.598168 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] 10.222.0.82 "PUT /v2.0/routers/d1ce813e-f0ed-493f-8885-b9884ac8f21e/add_router_interface HTTP/1.1" status: 200 len: 503 time: 10.7179320 May 25 16:03:20.681742 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Network 0105f026-2520-418a-a172-829f4b156413 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:20.693572 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:20.712086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:20.712893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.737462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Transition to ACTIVE for port object 8c1bac9a-e2d6-43eb-9551-100ddcab0607 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:20.864743 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:20.889416 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Pushing event updated for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:20.895928 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.418s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:21.059431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dacdd53f-099a-448f-99c1-53f4576d7923 req-6e10731d-c420-4469-a4ce-caac7ad7c252 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:21.064392 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dacdd53f-099a-448f-99c1-53f4576d7923 req-6e10731d-c420-4469-a4ce-caac7ad7c252 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.3222730 May 25 16:03:21.182511 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:21.197536 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:21.268751 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:21.376119 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dacdd53f-099a-448f-99c1-53f4576d7923 req-d35aba8a-6106-4f91-bbe5-8a0a63521aba tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2920904 May 25 16:03:21.410682 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:11Z', 'revision_number': 3, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:21.416673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3574dc1b-50a0-4229-b968-43bc20214750 None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:11Z', 'revision_number': 3, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:21.458019 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-930af18d-35d1-4297-839a-04948e83e9d1 None None] Provisioning for port 709f3425-4852-48b5-bffc-a7829c94b719 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:21.459869 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:21.463467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-930af18d-35d1-4297-839a-04948e83e9d1 None None] Provisioning complete for port 709f3425-4852-48b5-bffc-a7829c94b719 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:21.463945 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-930af18d-35d1-4297-839a-04948e83e9d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:21.558166 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 8e6bec27-45dc-4320-bf1a-16e932839558 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:21.797029 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-930af18d-35d1-4297-839a-04948e83e9d1 None None] Port 709f3425-4852-48b5-bffc-a7829c94b719 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:21.809747 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Pushing event updated for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:21.819742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.637s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:21.836119 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c9bccb0a-af4f-4c4d-93dc-5c2858c480c1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.328s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:21.844130 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 8e6bec27-45dc-4320-bf1a-16e932839558 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:21.844714 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:22.216993 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6a55392b-bd49-4082-889d-c7f77ed50499,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:22.244520 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Allocated IP 172.27.2.163 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/71ac5ece-56ff-4707-a295-aac78d41f825) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:22.409950 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:14Z', 'revision_number': 4, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '9ca03cbe-10dd-4e7e-a9ef-5a36837d3215', 'name': '', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'mac_address': 'fa:16:3e:c1:e3:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:12Z', 'updated_at': '2021-05-25T16:03:15Z', 'revision_number': 2, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'subnets': [{'id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:22.423645 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Checking router: d1ce813e-f0ed-493f-8885-b9884ac8f21e for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:22.424013 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Updating router d1ce813e-f0ed-493f-8885-b9884ac8f21e port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:22.469994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 3935ec55-2a13-43b8-b0bf-f9ef947051dc {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:22.480990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:22.507875 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Transition to ACTIVE for port object 3935ec55-2a13-43b8-b0bf-f9ef947051dc will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:22.520682 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.681s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:22.557281 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Attempting to bind port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:22.557281 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Attempting to bind port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'fcbe700b-26f1-40db-9201-5ec1c3805964', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:22.557281 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Attempting to bind port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 on network 5e540876-3bc6-4d60-aed5-9bca9dafe63d {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:22.565376 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 on network 5e540876-3bc6-4d60-aed5-9bca9dafe63d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:22.596071 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Attempting to bind port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 on network 5e540876-3bc6-4d60-aed5-9bca9dafe63d {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:22.596071 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:22.596071 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/6a55392b-bd49-4082-889d-c7f77ed50499 HTTP/1.1" status: 204 len: 168 time: 3.2474835 May 25 16:03:22.596424 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:22.606981 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:22.607560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Checking segment: {'id': 'fcbe700b-26f1-40db-9201-5ec1c3805964', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:22.608575 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Bound using segment: {'id': 'fcbe700b-26f1-40db-9201-5ec1c3805964', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:22.609261 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Bound port: 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'fcbe700b-26f1-40db-9201-5ec1c3805964', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:22.610529 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:22.746165 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:22.750203 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607,segment=NetworkSegment(fcbe700b-26f1-40db-9201-5ec1c3805964),segment_id=fcbe700b-26f1-40db-9201-5ec1c3805964)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:22.941909 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ddfb24f-a56c-4e34-84e3-42ede449ba11 req-52112d09-4682-46f4-901c-9cf18630cbdd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:22.952044 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2ddfb24f-a56c-4e34-84e3-42ede449ba11 req-52112d09-4682-46f4-901c-9cf18630cbdd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 932 time: 1.2186544 May 25 16:03:22.963731 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.030373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-eaecf704-69d4-4bf8-b6ee-eb25c6b66488 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:23.041264 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-eaecf704-69d4-4bf8-b6ee-eb25c6b66488 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 818 time: 1.1760831 May 25 16:03:23.047950 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1dfa54c9-e587-466e-ab83-1d752d90bfb1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:23.098521 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2ddfb24f-a56c-4e34-84e3-42ede449ba11 req-65ed49fe-61bd-4345-9250-bda5fed9e063 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1434228 May 25 16:03:23.124297 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-a7ed0bc3-f072-44c7-b31e-a441a770d9f9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/quotas/d20a1b70ac3f4086aac2870ee8c08efe HTTP/1.1" status: 200 len: 375 time: 0.0821097 May 25 16:03:23.250499 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.279342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-e344405c-2b67-42b7-b482-62a76b12b6a0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Synchronizing usage tracker for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:23.486928 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.496741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-162009cf-500e-4cfc-a40e-ba7aa0d5ec22 req-d2d84e1a-a2d1-4215-9feb-ae8587245ec9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:23.504830 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-162009cf-500e-4cfc-a40e-ba7aa0d5ec22 req-d2d84e1a-a2d1-4215-9feb-ae8587245ec9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.1280222 May 25 16:03:23.524866 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device e2560d2e-a52a-4dbc-a74c-eda92a478df8 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:23.565380 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.573474 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-162009cf-500e-4cfc-a40e-ba7aa0d5ec22 req-22d2ee5b-f84c-48f7-b7ea-f9e42747f825 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0677547 May 25 16:03:23.576150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.599362 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9240382a-dbce-41da-9dd0-17653eb00579 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:23.668031 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.673549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.717215 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-e344405c-2b67-42b7-b482-62a76b12b6a0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Unset dirty status for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:23.717215 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-e344405c-2b67-42b7-b482-62a76b12b6a0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&fields=id HTTP/1.1" status: 200 len: 255 time: 0.5826859 May 25 16:03:23.814702 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:23.818893 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:23.819643 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.821306 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] 10.222.0.42 "PUT /v2.0/ports/8c1bac9a-e2d6-43eb-9551-100ddcab0607 HTTP/1.1" status: 200 len: 1093 time: 5.9422643 May 25 16:03:23.822629 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:23.824888 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:23.896565 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:23.904444 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port e2560d2e-a52a-4dbc-a74c-eda92a478df8 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:23.920009 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port e2560d2e-a52a-4dbc-a74c-eda92a478df8 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:23.924928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:23.927286 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:23.942984 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.896s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:23.947610 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.241s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:24.072440 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Pushing event updated for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:24.103053 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:24.142419 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Transition to ACTIVE for port object 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:24.148857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.551s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:24.159839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Attempting to bind port 3935ec55-2a13-43b8-b0bf-f9ef947051dc on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:24.164040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Attempting to bind port 3935ec55-2a13-43b8-b0bf-f9ef947051dc by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:24.164040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Attempting to bind port 3935ec55-2a13-43b8-b0bf-f9ef947051dc on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:24.357824 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:24.413223 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:24.575842 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:24.620876 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port e2560d2e-a52a-4dbc-a74c-eda92a478df8, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=e2560d2e-a52a-4dbc-a74c-eda92a478df8,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:24.680918 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Port 3935ec55-2a13-43b8-b0bf-f9ef947051dc on network 9123c0ac-e64c-41a9-8d68-d98246284959 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:24.681583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Attempting to bind port 3935ec55-2a13-43b8-b0bf-f9ef947051dc on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:24.717072 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:24.720757 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:24.764193 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:24.764534 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Checking segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:24.764931 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Bound using segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:24.765381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Bound port: 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:24.765732 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:24.841224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7dae5c68-c6c1-4e39-97e1-48bd29bdf0eb service neutron] Pushing event updated for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:24.849729 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-6b913ccf-0033-4fc8-9704-cdbfe6e24f32 service neutron] 10.222.0.82 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 1.1007488 May 25 16:03:24.863837 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Network 0105f026-2520-418a-a172-829f4b156413 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:24.868354 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:24.910542 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-201cfdfb-078c-47a2-a0fa-cbb4502247bd req-d1da04ec-83b9-493a-99eb-3a3dbbcf841f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:24.912466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-201cfdfb-078c-47a2-a0fa-cbb4502247bd req-d1da04ec-83b9-493a-99eb-3a3dbbcf841f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.2820461 May 25 16:03:25.007106 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=e2560d2e-a52a-4dbc-a74c-eda92a478df8,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:25.014860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.294s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:25.027990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.104975 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Attempting to bind port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:25.106928 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Attempting to bind port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:25.107680 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Attempting to bind port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:25.134505 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:25.135355 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Checking segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:25.136870 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.137918 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Bound using segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:25.138647 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Bound port: 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:25.139417 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.246584 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:25.251536 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:25.386747 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14e9d0d2-cb28-45f6-af13-579e1b1fb576 None None] Pushing event updated for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:25.393764 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-14e9d0d2-cb28-45f6-af13-579e1b1fb576 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.982s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:25.395566 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-85097303-ef3e-4b1e-86a6-505ed475c435 None None] Provisioning complete for port 709f3425-4852-48b5-bffc-a7829c94b719 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:25.397909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-85097303-ef3e-4b1e-86a6-505ed475c435 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.400931 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] For port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:25.408632 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] For port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:25.441994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.487031 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.492831 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device e2560d2e-a52a-4dbc-a74c-eda92a478df8 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:25.492940 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:25.534439 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=71ac5ece-56ff-4707-a295-aac78d41f825,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:25.541122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 1.593s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:25.541122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.606785 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-201cfdfb-078c-47a2-a0fa-cbb4502247bd req-ba0efd62-c6fb-4d0f-847b-8c9b4c52676c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2054527 May 25 16:03:25.653191 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-14e9d0d2-cb28-45f6-af13-579e1b1fb576 None None] Pushing event updated for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:25.679881 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-14e9d0d2-cb28-45f6-af13-579e1b1fb576 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.187s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:25.704063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 1.899s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:25.723296 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:25.733979 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:25.756864 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:03:25.785583 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-0c684823-5905-430a-b6c2-0b3794f229a8 service neutron] 10.222.0.82 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.9189117 May 25 16:03:25.830904 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-85097303-ef3e-4b1e-86a6-505ed475c435 None None] Port 709f3425-4852-48b5-bffc-a7829c94b719 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:25.862352 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port e2560d2e-a52a-4dbc-a74c-eda92a478df8 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:25.862679 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.930306 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:25.930951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:25.932369 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] 10.222.0.42 "PUT /v2.0/ports/3935ec55-2a13-43b8-b0bf-f9ef947051dc HTTP/1.1" status: 200 len: 1094 time: 5.6552749 May 25 16:03:26.094635 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-0a94d1bc-0c88-4471-89f9-86ebd2d9a634 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:26.096221 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-0a94d1bc-0c88-4471-89f9-86ebd2d9a634 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=96b2d1f8431044fca9997bb410be55b8&device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 1096 time: 2.0598407 May 25 16:03:26.187676 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:03:26.213719 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] 10.222.0.82 "PUT /v2.0/routers/b1fcb15a-38af-4be8-95c4-8843c2278f7c/add_router_interface HTTP/1.1" status: 200 len: 503 time: 9.8569558 May 25 16:03:26.229217 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:03:26.249124 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:26.306120 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:26.323063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.619s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:26.325323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.821s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:26.325972 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:26.328681 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-03c3d802-b27c-4ed2-ad63-d65c5e1eae9b service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607 HTTP/1.1" status: 200 len: 217 time: 0.1606102 May 25 16:03:26.362978 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:26.363939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:26.654606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd3e6466-3e44-414b-9c0e-83557e142c0c req-cb81bc76-1360-4020-a3f5-c7046df18a81 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:26.656253 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dd3e6466-3e44-414b-9c0e-83557e142c0c req-cb81bc76-1360-4020-a3f5-c7046df18a81 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.5101523 May 25 16:03:26.779368 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dd3e6466-3e44-414b-9c0e-83557e142c0c req-5029d03c-6889-4656-9aae-86c78bed6326 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0869100 May 25 16:03:26.980732 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:26.999794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-54a79d46-00e7-46e5-b688-7a08f73e8001 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:27.004311 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-54a79d46-00e7-46e5-b688-7a08f73e8001 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=7a4c9393-23ba-4625-b1b4-895efa8276f1 HTTP/1.1" status: 200 len: 851 time: 0.6652677 May 25 16:03:27.049461 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d783c3b-6b4e-46eb-9038-57e1d6e329da req-76d005e2-1cf0-413a-bd63-e5198233faca tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.1806448 May 25 16:03:27.153026 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-90b6ca8b-d388-494d-bc80-3dd6ad25c1ee service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=5e540876-3bc6-4d60-aed5-9bca9dafe63d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1208465 May 25 16:03:27.347991 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:14Z', 'revision_number': 4, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '9ca03cbe-10dd-4e7e-a9ef-5a36837d3215', 'name': '', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'mac_address': 'fa:16:3e:c1:e3:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:12Z', 'updated_at': '2021-05-25T16:03:25Z', 'revision_number': 4, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'subnets': [{'id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:27.358076 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2d9a90da-6fd9-4f57-bd27-4af912c0407b None None] The sync data for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi is: [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:14Z', 'revision_number': 4, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '9ca03cbe-10dd-4e7e-a9ef-5a36837d3215', 'name': '', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'mac_address': 'fa:16:3e:c1:e3:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_interface', 'standard_attr_id': 100, 'fixed_ips': [{'subnet_id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:12Z', 'updated_at': '2021-05-25T16:03:25Z', 'revision_number': 4, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'subnets': [{'id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:27.456153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-1a04f0e8-b866-4587-85ef-362ab43c6d6a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:27.468583 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-1a04f0e8-b866-4587-85ef-362ab43c6d6a service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 1097 time: 0.8349416 May 25 16:03:27.695394 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607', 'status': 'completed', 'code': 200} May 25 16:03:27.706321 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-0f1ab008-1318-4401-b5e5-bc10c39a926d service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5276611 May 25 16:03:27.707129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40572) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:27.716907 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-e533262d-ae8c-477f-8433-13836e3b679c service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc HTTP/1.1" status: 200 len: 217 time: 0.2397470 May 25 16:03:27.718288 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.db.resource_extend [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] It took 0.26 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 25 16:03:28.118342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-58f42518-0008-40b3-b17c-56786ac9d62a req-85db12db-efcc-43ad-8402-f95b27aec771 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:28.139058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-58f42518-0008-40b3-b17c-56786ac9d62a req-85db12db-efcc-43ad-8402-f95b27aec771 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.2704527 May 25 16:03:28.147794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.167242 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 2, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:28.185155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:03:28.185728 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Updating router b1fcb15a-38af-4be8-95c4-8843c2278f7c port 709f3425-4852-48b5-bffc-a7829c94b719 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:03:28.243822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.247688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.293845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:28.311883 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Usage tracker for resource:security_group_rule and tenant:f9a447e7dde44d1aa698b50e5d0da25d is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:28.317050 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:28.339149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:28.383520 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.390176 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b4ab6edd-c3e9-437e-8c04-d6ff7728ddf1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:28.522292 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-4df2b94b-84b7-4328-bc47-216961679407 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:28.523884 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-4df2b94b-84b7-4328-bc47-216961679407 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.5875704 May 25 16:03:28.606469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.726348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.804547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.820227 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 10.6566420 May 25 16:03:28.829696 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-b1c8eceb-684c-4a6b-a287-eeebb78a63f7 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.2717671 May 25 16:03:28.838292 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-58f42518-0008-40b3-b17c-56786ac9d62a req-5bfd434f-d625-48ee-83cb-0954c1045f2f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.6979825 May 25 16:03:28.844144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:28.851982 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-25dfada9-7cf1-4a17-b284-4638399f4910 req-d7901786-9c6e-4461-9da6-844958ece484 service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 1.1049097 May 25 16:03:28.914264 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:28.959989 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=71ac5ece-56ff-4707-a295-aac78d41f825,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:28.977576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.596s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:29.086320 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40602) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:29.117073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group': {'name': 'tempest-securitygroup--1265063046', 'description': 'tempest-description--1524718573'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:29.223528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:29.241402 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:29.263316 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:29.281121 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:29.305467 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f9a447e7dde44d1aa698b50e5d0da25d&name=default HTTP/1.1" status: 200 len: 2979 time: 1.5966988 May 25 16:03:29.319768 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f9a5caa8-032a-4a77-94f9-0a56ac3a1184,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:29.330974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40616) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:29.335401 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.423s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:29.545964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:29.553526 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:29.581227 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:29.709211 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Transition to ACTIVE for port object 709f3425-4852-48b5-bffc-a7829c94b719 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:29.720241 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:29.833339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:29.837180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:29.909848 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-d03bbfd8-d814-4d7e-abc2-1962e6bddf1d service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.9990537 May 25 16:03:29.910641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:29.916794 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:29.920173 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:29.922208 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f421d99-c8b3-464d-983c-eabe439768b2 None None] Pushing event updated for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:29.933585 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-6f421d99-c8b3-464d-983c-eabe439768b2 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:29.941820 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-97d79fdc-471c-49e2-8aff-334b64dc178e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:29.942594 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-97d79fdc-471c-49e2-8aff-334b64dc178e service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=96b2d1f8431044fca9997bb410be55b8&device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 1096 time: 0.7155321 May 25 16:03:29.956199 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:29.964174 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:30.077089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.080994 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:30.114689 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:30.115102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.192046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3eb0216f-ddb8-4d8a-8e4c-756a7ed50b5c req-ac128d59-4a8b-4ba6-9b46-2a1f3b1fe10c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:30.196484 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3eb0216f-ddb8-4d8a-8e4c-756a7ed50b5c req-ac128d59-4a8b-4ba6-9b46-2a1f3b1fe10c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.6553206 May 25 16:03:30.277480 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-8571c8ad-3b6a-4859-b26f-f1ef6ee4d40e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:30.278616 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0404c32f-f559-48fe-8b05-c254a0e362e1 req-814f1824-f19c-4015-b1a4-a5ea0b038745 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2373779 May 25 16:03:30.280035 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-8571c8ad-3b6a-4859-b26f-f1ef6ee4d40e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 818 time: 1.1051326 May 25 16:03:30.409475 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0973da2-ad34-4d55-8f65-ae824ada8c5f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/quotas/d20a1b70ac3f4086aac2870ee8c08efe HTTP/1.1" status: 200 len: 375 time: 0.0641608 May 25 16:03:30.466703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Attempting to bind port 709f3425-4852-48b5-bffc-a7829c94b719 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:30.467270 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Attempting to bind port 709f3425-4852-48b5-bffc-a7829c94b719 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:30.471421 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Attempting to bind port 709f3425-4852-48b5-bffc-a7829c94b719 on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Checking segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Bound using segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Bound port: 709f3425-4852-48b5-bffc-a7829c94b719, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:30.486274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.498854 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3eb0216f-ddb8-4d8a-8e4c-756a7ed50b5c req-963616eb-4090-43de-912d-27870c69c44a tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2468960 May 25 16:03:30.535193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.569235 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-8f1b5668-9d9e-45fd-9843-4702cc1f2995 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:30.570818 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-8f1b5668-9d9e-45fd-9843-4702cc1f2995 service neutron] 10.222.0.42 "GET /v2.0/networks?id=5e540876-3bc6-4d60-aed5-9bca9dafe63d HTTP/1.1" status: 200 len: 912 time: 0.2972558 May 25 16:03:30.589476 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-83fde0b1-88fd-4654-89f1-c42f7d4ba742 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&fields=id HTTP/1.1" status: 200 len: 255 time: 0.1668119 May 25 16:03:30.684650 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.725192 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:30.742219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] For port 709f3425-4852-48b5-bffc-a7829c94b719, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:30.747395 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] For port 709f3425-4852-48b5-bffc-a7829c94b719, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=709f3425-4852-48b5-bffc-a7829c94b719,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:30.750441 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-7e5aa677-3f64-4647-a304-3ca036894ddd service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607 HTTP/1.1" status: 200 len: 217 time: 0.1299851 May 25 16:03:30.774032 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.790271 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.796826 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca03fb8f-f1ca-4a2c-9392-b0b77be27151 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:30.819940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-eb284899-a9c1-4f0f-a8a4-4a97bd32c7a5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:30.821238 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f9a5caa8-032a-4a77-94f9-0a56ac3a1184,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:30.821726 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-eb284899-a9c1-4f0f-a8a4-4a97bd32c7a5 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=7a4c9393-23ba-4625-b1b4-895efa8276f1 HTTP/1.1" status: 200 len: 851 time: 0.0631618 May 25 16:03:30.877393 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.903742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.912398 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:30.922302 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-1030c92a-e5e3-4fd8-8e40-04b6a71005f4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:30.942747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:30.943869 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d0fe58f6-4af3-49d1-a7f1-d6b68dae0022 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f9a5caa8-032a-4a77-94f9-0a56ac3a1184 HTTP/1.1" status: 204 len: 168 time: 1.6075046 May 25 16:03:30.965206 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-185dae43-d9d9-4460-a510-b5b99bdc7a5a service neutron] 10.222.0.82 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3745611 May 25 16:03:30.986468 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroup': ['ID=e4aeeac6-6179-49e9-a625-03f354d51988,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:30.987633 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-4a8ed691-43b0-4ed9-a880-df40c55cb354 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=5e540876-3bc6-4d60-aed5-9bca9dafe63d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0636187 May 25 16:03:30.994147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:30.998360 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.179s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:30.998360 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:31.033446 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f421d99-c8b3-464d-983c-eabe439768b2 None None] Pushing event updated for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:31.065606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.083730 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1780 time: 1.9700429 May 25 16:03:31.088903 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40668) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:31.095812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group_rule': {'security_group_id': 'e4aeeac6-6179-49e9-a625-03f354d51988', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:31.111476 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:31.116061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-1030c92a-e5e3-4fd8-8e40-04b6a71005f4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:31.117279 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-1030c92a-e5e3-4fd8-8e40-04b6a71005f4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-securitygroup--1686244828&fields=id&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 267 time: 0.3635964 May 25 16:03:31.132779 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:31.133869 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:31.139229 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.227s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:31.140164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:31.141776 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.137s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:31.142143 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:31.153235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:03:31.154036 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 2, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 1, "log_agent_heartbeats": false, "routers": 2},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:31.198438 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:31.394367 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.398716 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-bf7ddf60-4256-48b5-b219-49759842939f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:31.399545 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-bf7ddf60-4256-48b5-b219-49759842939f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Synchronizing usage tracker for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:network {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:31.413226 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:31.457136 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:03:31.520294 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-2d7774f4-6164-4b61-8469-217db70ef08b service neutron] 10.222.0.82 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5533712 May 25 16:03:31.523372 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.546424 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-186e261f-3869-46ee-9b32-64b6ed9ce8a6 service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5223482 May 25 16:03:31.619983 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6e627af0-d33b-440e-acf5-9047565d9327 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:31.620812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6e627af0-d33b-440e-acf5-9047565d9327 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:31.700696 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:31.700696 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.757219 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e4d50ce5-cd1f-4d97-8b59-99e19edbaa42 req-2d37bf81-1273-46e0-b9da-220609682789 service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2031126 May 25 16:03:31.783969 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.794205 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:31.797763 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-bf7ddf60-4256-48b5-b219-49759842939f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Unset dirty status for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:network {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:31.799630 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-bf7ddf60-4256-48b5-b219-49759842939f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 818 time: 0.7740717 May 25 16:03:31.821306 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Request body: {'port': {'device_id': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'admin_state_up': True, 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:31.836573 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=8f3caeef-3309-44db-a7a9-20575d819584,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:31.843039 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6e627af0-d33b-440e-acf5-9047565d9327 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:31.856920 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6e627af0-d33b-440e-acf5-9047565d9327 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 817 time: 0.7180729 May 25 16:03:31.875155 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:31.896890 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-708f2747-0f51-412d-92fc-96fa66ed085a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/quotas/690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 375 time: 0.0339706 May 25 16:03:31.976626 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:31.977991 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.8875885 May 25 16:03:31.984293 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40686) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:31.991718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group_rule': {'security_group_id': 'e4aeeac6-6179-49e9-a625-03f354d51988', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:32.000371 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'status': 'completed', 'code': 200} May 25 16:03:32.001429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:32.008122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:32.013375 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:32.021288 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:32.095018 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-5a3b0a5c-7e77-4c6a-85c8-c673cc8c3c68 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1992514 May 25 16:03:32.180374 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:30Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:03:32.210861 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-05fd1be5-afbe-492f-a5cc-7388702c074e None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:30Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:03:32.259419 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:32.287079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:32.319766 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-fea5391c-479b-46fa-a651-1f3998c3b1e8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:32.321217 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-fea5391c-479b-46fa-a651-1f3998c3b1e8 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 1097 time: 0.2416625 May 25 16:03:32.353363 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40704) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:32.443508 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.506133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Made reservation on behalf of d20a1b70ac3f4086aac2870ee8c08efe for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:32.513503 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.611678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40712) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:32.670569 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.681394 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.691912 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-f8b28bb6-1542-458d-8b42-8a8c8628b496 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:32.693764 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-f8b28bb6-1542-458d-8b42-8a8c8628b496 service neutron] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 915 time: 0.3652332 May 25 16:03:32.709290 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-e1ac4fb2-efcd-4439-9dcc-1c95060ea161 service neutron] 10.222.0.82 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5915973 May 25 16:03:32.713607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.716515 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:32.748078 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3803a1ec-e993-4e3f-aa26-0bc5d0f16c05 req-72f30aca-999c-46d2-9701-1aa199f3bdc3 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:32.753680 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3803a1ec-e993-4e3f-aa26-0bc5d0f16c05 req-72f30aca-999c-46d2-9701-1aa199f3bdc3 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.4547806 May 25 16:03:32.753680 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Usage tracker for resource:security_group_rule and tenant:7c9e2de3c8164e2f8ed91afbfe6b41cd is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:32.755905 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:32.858893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:32.879364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:32.887384 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-abd8f794-7b6d-4e90-881e-d8ae3a03600f service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc HTTP/1.1" status: 200 len: 217 time: 0.1874943 May 25 16:03:32.897508 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Usage tracker for resource:security_group_rule and tenant:882d90218491415e82ada1ee09dc552b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:32.897733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:32.908134 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:32.923262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-624ea002-e094-4cdb-97bd-957148f25a24 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:32.951962 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:32.990445 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3803a1ec-e993-4e3f-aa26-0bc5d0f16c05 req-3cca7ce7-4537-4cbb-9013-66ce55d302a2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2295752 May 25 16:03:32.999622 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=24f0d585-9b7a-4054-83a0-28b8c599c9c2,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:33.012542 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:33.045557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-3f72a794-d4c7-49bf-a661-6d74ad41f7cb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:33.049057 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-3f72a794-d4c7-49bf-a661-6d74ad41f7cb service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.1472938 May 25 16:03:33.065283 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-f937a9e6-c423-43ca-9017-89e601f12efc service neutron] 10.222.0.82 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3451169 May 25 16:03:33.092613 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Allocated IP 172.20.0.4 (feb61d72-e6b4-48c6-bc7c-d61f51987cf9/051377f2-6b21-4071-b8f5-21107e3c870f/76801a33-2412-4d01-aee4-406f117edbd1) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:33.141261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.145405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.151241 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-962cc86b-5e02-4efb-9709-cb6f64d94873 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 1.1607053 May 25 16:03:33.151241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40718) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:33.157418 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:33.185751 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:33.200969 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-56c6114a-6f39-4d9b-bdf9-3f137039efb7 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1161194 May 25 16:03:33.204498 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:floatingip and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:33.210419 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:33.268257 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:03:33.272802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:33.280712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.300495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:33.350154 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=7c9e2de3c8164e2f8ed91afbfe6b41cd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7247231 May 25 16:03:33.355158 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40722) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:33.371315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d1f17c89-a0d5-4822-9285-6ee662c72e33,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:33.377978 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:33.425262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6700c649-a511-4c59-9ad1-6741f7ec6989 req-a8fb22c4-76e1-42ed-96b0-891e39891e0c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:33.426926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6700c649-a511-4c59-9ad1-6741f7ec6989 req-a8fb22c4-76e1-42ed-96b0-891e39891e0c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.2180269 May 25 16:03:33.476539 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.489929 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 709f3425-4852-48b5-bffc-a7829c94b719 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:33.533895 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.655389 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6700c649-a511-4c59-9ad1-6741f7ec6989 req-f41b478d-e364-4907-9a19-52310792a0f9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2113569 May 25 16:03:33.691996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.746687 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 709f3425-4852-48b5-bffc-a7829c94b719 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:33.753202 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 709f3425-4852-48b5-bffc-a7829c94b719 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:33.753592 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.819308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:33.895089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fb03901c-13bf-4a32-b5ef-f2ccac461a28 req-c0a27a1d-9342-4b6c-8e4b-3d73f88dd76c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.1619136 May 25 16:03:33.905089 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-cd7ac0ac-ecfd-4b97-8b42-32f7e1628b82 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.6882272 May 25 16:03:33.958444 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ca870f73-df00-4aaa-b8a6-3135e6f0ba4b req-f4718c2d-94d4-4ea2-b095-123cddc33b94 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 210 time: 0.1366792 May 25 16:03:33.965598 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.024042 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 709f3425-4852-48b5-bffc-a7829c94b719, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=709f3425-4852-48b5-bffc-a7829c94b719,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:34.064905 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.116639 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.143961 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.147826 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.172125 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.182141 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1e94b0eb-33a5-4bc7-8072-7e15a782c0db req-a5bc3bdb-5516-4be0-869d-aa1a071e3fdb service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2537434 May 25 16:03:34.219074 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.224577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:34.225704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.232240 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:34.235220 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.285354 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=882d90218491415e82ada1ee09dc552b&name=default HTTP/1.1" status: 200 len: 2979 time: 1.9309180 May 25 16:03:34.286340 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=079471d5-8c74-40b7-b616-81e354e4c562,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:34.290067 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:34.291232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40746) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:34.298760 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.302742 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d1f17c89-a0d5-4822-9285-6ee662c72e33,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:34.321727 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:34.339494 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.340636 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/d1f17c89-a0d5-4822-9285-6ee662c72e33 HTTP/1.1" status: 204 len: 168 time: 0.9845557 May 25 16:03:34.368164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Allocated IP 172.27.2.179 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/3a0d7127-7f88-407f-9fd4-348a20a45471) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:34.386360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.477279 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:34.482977 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:34.487789 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.489703 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:34.496093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:34.496809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.501759 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Pushing event updated for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:34.504658 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a01a2730-44ad-442e-a0b1-c3c22ed4d00b req-2cb84180-e748-4c4d-a235-8834a706eb02 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:34.522778 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a01a2730-44ad-442e-a0b1-c3c22ed4d00b req-2cb84180-e748-4c4d-a235-8834a706eb02 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.3459575 May 25 16:03:34.522896 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:34.544989 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.597283 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a01a2730-44ad-442e-a0b1-c3c22ed4d00b req-da5d219f-cb4c-45a7-8e72-7343410f518b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0850306 May 25 16:03:34.606131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.716318 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.9000108 May 25 16:03:34.735152 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Request body: {'port': {'device_id': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:34.772935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.786754 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.802247 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.813405 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:34.833408 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.834575 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/079471d5-8c74-40b7-b616-81e354e4c562 HTTP/1.1" status: 204 len: 168 time: 0.5423815 May 25 16:03:34.838199 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=079471d5-8c74-40b7-b616-81e354e4c562,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:34.850990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:34.921373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:34.930647 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:34.940913 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:34.941789 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:34.975939 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=3a0d7127-7f88-407f-9fd4-348a20a45471,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:35.045511 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.265s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:35.112470 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-26ea6c6f-403a-4895-8d29-3c597d7fd385 req-94f43309-d387-4d68-aaff-6d13b92c1abd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.0659590 May 25 16:03:35.134109 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:35.157276 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1d9761e-027d-4d46-b805-77c498f4f510 req-31bc4d63-1ff7-4301-9024-9be57a111428 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:35.167969 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1d9761e-027d-4d46-b805-77c498f4f510 req-31bc4d63-1ff7-4301-9024-9be57a111428 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.3016014 May 25 16:03:35.178219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 3935ec55-2a13-43b8-b0bf-f9ef947051dc details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:35.249067 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1d9761e-027d-4d46-b805-77c498f4f510 req-82b41dd4-5a6c-475e-8ebf-a6cc59457cdd tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0834589 May 25 16:03:35.339717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:35.442061 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 709f3425-4852-48b5-bffc-a7829c94b719 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:35.615704 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:35.634163 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'port_id': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'mac_address': 'fa:16:3e:b4:aa:cb', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 948, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b19f7657-6038-4479-b9ec-9e4e92c69bb0', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:35.790149 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 709f3425-4852-48b5-bffc-a7829c94b719 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:35.795910 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:35.842381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-6fda9b32-4040-4ea6-a559-7d196a39d62a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:35.845470 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-6fda9b32-4040-4ea6-a559-7d196a39d62a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 818 time: 0.8276045 May 25 16:03:35.882834 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Request body: {'port': {'device_id': '43046111-0b2a-499f-8353-b8fd77b989ea', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'admin_state_up': True, 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:35.937878 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-071ad697-2a2f-4ae8-a0d7-9de6757bd459 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:35.949874 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:35.972261 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 3935ec55-2a13-43b8-b0bf-f9ef947051dc {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:35.994183 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:36.088649 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-918bb7c8-019a-4696-80de-4c86560e7861 req-559b70e3-5fad-41cd-8259-45c8f7750cf8 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 210 time: 0.1742291 May 25 16:03:36.222820 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-27d5e0c2-9f39-4409-8b7a-058a5ae1d604 req-3ba91bd5-184d-4b27-abc6-0a2757f140d2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:36.224748 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-27d5e0c2-9f39-4409-8b7a-058a5ae1d604 req-3ba91bd5-184d-4b27-abc6-0a2757f140d2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.3083060 May 25 16:03:36.227887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.281498 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.285382 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 8c1bac9a-e2d6-43eb-9551-100ddcab0607 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:36.338540 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-27d5e0c2-9f39-4409-8b7a-058a5ae1d604 req-09bd006a-527a-4d95-b47c-966bb7b78a9e tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0879230 May 25 16:03:36.340391 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:36.358940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Usage tracker for resource:port and tenant:d20a1b70ac3f4086aac2870ee8c08efe is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:36.364552 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:36.394664 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.399328 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:36.549435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40814) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:36.583656 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-704407d3-8a1c-4b74-9ac6-3264aa0980f4 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:36.593382 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:36.659597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 76801a33-2412-4d01-aee4-406f117edbd1 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:36.677186 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.696328 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Transition to ACTIVE for port object 76801a33-2412-4d01-aee4-406f117edbd1 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:36.712907 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:36.724760 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:36.738039 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Made reservation on behalf of d20a1b70ac3f4086aac2870ee8c08efe for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:36.739124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.748990 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607,segment=NetworkSegment(fcbe700b-26f1-40db-9201-5ec1c3805964),segment_id=fcbe700b-26f1-40db-9201-5ec1c3805964)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:36.768852 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '8c1bac9a-e2d6-43eb-9551-100ddcab0607', 'network_id': '5e540876-3bc6-4d60-aed5-9bca9dafe63d', 'port_id': '8c1bac9a-e2d6-43eb-9551-100ddcab0607', 'mac_address': 'fa:16:3e:2b:a3:26', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 901, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '7a4c9393-23ba-4625-b1b4-895efa8276f1', 'ip_address': '172.20.0.2'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:36.809165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.830624 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.833712 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.849137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:36.854216 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 8c1bac9a-e2d6-43eb-9551-100ddcab0607 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:36.877479 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607,segment=NetworkSegment(fcbe700b-26f1-40db-9201-5ec1c3805964),segment_id=fcbe700b-26f1-40db-9201-5ec1c3805964)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:36.888830 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:36.902673 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3bd54f60-bfaf-474d-9078-abb5a110ef67 req-e57c092c-d07b-40c5-80af-1a7e78ce5177 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.3597400 May 25 16:03:36.924131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:36.928534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Usage tracker for resource:security_group_rule and tenant:dedae466be174d7e8ed1dd820d0c600d is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:36.937788 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:36.937788 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:36.980874 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=3a0d7127-7f88-407f-9fd4-348a20a45471,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:36.991941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.593s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:37.006449 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.008277 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:37.008948 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.010315 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 3.8593922 May 25 16:03:37.083403 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dca8abd3-44c1-4345-8de2-ca60393c5799 req-7c701291-1aeb-4062-8db9-5cb37fc7880e tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:37.085061 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dca8abd3-44c1-4345-8de2-ca60393c5799 req-7c701291-1aeb-4062-8db9-5cb37fc7880e tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 943 time: 0.3756590 May 25 16:03:37.218739 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Pushing event updated for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:37.246152 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.397s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:37.249607 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Attempting to bind port 76801a33-2412-4d01-aee4-406f117edbd1 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:37.250034 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Attempting to bind port 76801a33-2412-4d01-aee4-406f117edbd1 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:37.250361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Attempting to bind port 76801a33-2412-4d01-aee4-406f117edbd1 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:37.252370 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:37.267892 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.259s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:37.271296 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Port 76801a33-2412-4d01-aee4-406f117edbd1 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:37.271845 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Attempting to bind port 76801a33-2412-4d01-aee4-406f117edbd1 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:37.280105 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Allocated IP 172.20.0.2 (feb61d72-e6b4-48c6-bc7c-d61f51987cf9/051377f2-6b21-4071-b8f5-21107e3c870f/a7ff877f-0a9b-46af-9cc4-578eeaaa2e89) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:37.286555 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:37.286924 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Checking segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:37.287314 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Bound using segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:37.287813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Bound port: 76801a33-2412-4d01-aee4-406f117edbd1, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:37.288164 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.322667 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dca8abd3-44c1-4345-8de2-ca60393c5799 req-fcf4ee59-2321-47ff-b663-c694c2217cf5 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2303770 May 25 16:03:37.334382 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.411536 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:37.453368 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-48f39055-0b5e-40ad-8d03-84f96c824eb2 req-ef1a6163-9a3f-4a01-b2c7-6bbaac584bce tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 210 time: 0.0571315 May 25 16:03:37.468942 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['8c1bac9a-e2d6-43eb-9551-100ddcab0607'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:37.522632 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.548018 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:37.571695 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.42', 57087) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:37.579044 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=76801a33-2412-4d01-aee4-406f117edbd1,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:37.604881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:37.614595 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=dedae466be174d7e8ed1dd820d0c600d&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0460794 May 25 16:03:37.618439 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 40846) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:37.706503 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.731745 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:37.798050 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=67e010c0-fbcf-4e23-aef6-eef3e2681d74,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:37.804656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:37.831809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:37.832587 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.834000 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] 10.222.0.42 "PUT /v2.0/ports/76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 1093 time: 3.1043775 May 25 16:03:37.847089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-44656ffc-188a-4778-9325-bbbbd8df5317 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:37.853735 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:37.875413 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.897861 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.926397 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6856a0a4-f619-4e4d-a6c3-482d1c2cdb6c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:37.927901 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-6856a0a4-f619-4e4d-a6c3-482d1c2cdb6c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 817 time: 0.3535266 May 25 16:03:37.934970 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Pushing event updated for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:37.940172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5654146-3d79-4447-a55f-027d7e0003ca req-def94763-1afe-4948-b971-58cb905e8955 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:37.941857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e5654146-3d79-4447-a55f-027d7e0003ca req-def94763-1afe-4948-b971-58cb905e8955 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 945 time: 0.2539158 May 25 16:03:37.944018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:37.954560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.223s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:38.021815 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-44656ffc-188a-4778-9325-bbbbd8df5317 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:38.025605 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-26a4b8a3-1531-4504-95ce-bb6df111a35c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:38.029598 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-44656ffc-188a-4778-9325-bbbbd8df5317 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-securitygroup--1265063046&fields=id&tenant_id=06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 267 time: 0.4119081 May 25 16:03:38.052722 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-26a4b8a3-1531-4504-95ce-bb6df111a35c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:38.054481 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-26a4b8a3-1531-4504-95ce-bb6df111a35c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 380 time: 0.0989234 May 25 16:03:38.066457 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'port': {'device_id': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'admin_state_up': True, 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'security_groups': ['32ab79bb-f281-465f-93f1-440cbf7dec8c']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:38.081835 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e5654146-3d79-4447-a55f-027d7e0003ca req-636f33bd-05d0-4374-8282-1157ddf2b7f8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1292989 May 25 16:03:38.208411 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1', 'status': 'completed', 'code': 200} May 25 16:03:38.227146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.241736 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.269282 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:38.274884 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-2e42c07c-b109-453a-ac80-08f786b1d0e0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:38.277109 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-2e42c07c-b109-453a-ac80-08f786b1d0e0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:38.342669 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.347534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:38.381780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=67e010c0-fbcf-4e23-aef6-eef3e2681d74,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:38.386969 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:38.404813 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.408966 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/67e010c0-fbcf-4e23-aef6-eef3e2681d74 HTTP/1.1" status: 204 len: 168 time: 0.7829599 May 25 16:03:38.423994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:38.444721 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:38.466105 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:38.466355 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-2e42c07c-b109-453a-ac80-08f786b1d0e0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:38.473569 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-2e42c07c-b109-453a-ac80-08f786b1d0e0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 811 time: 0.4429789 May 25 16:03:38.474312 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:38.475218 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.487040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Pushing event updated for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:38.508768 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95763c5b-3fc0-445a-99f6-9e36ca37f751 req-721b10aa-a439-4e5b-8b0d-ca32f509afa8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.0846922 May 25 16:03:38.528748 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-03fec7e6-7cbb-4dac-8b07-dcea8328ea08 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/quotas/06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 375 time: 0.0525773 May 25 16:03:38.529138 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.260s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:38.531622 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-42ea10f5-896e-4006-80ba-f2ca4e1c22b7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.537547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:38.562298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:port and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:38.575137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for port was recalculated. Used quota:3. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:38.614303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:38.622308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-efe61104-11f1-4d7d-9b5f-2d092af19fca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:38.622308 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-efe61104-11f1-4d7d-9b5f-2d092af19fca service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1096 time: 0.5313563 May 25 16:03:38.685074 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.731040 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.8563521 May 25 16:03:38.740330 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-32044b01-a1c1-4c52-968a-cba0d4a1f716 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.1055279 May 25 16:03:38.756977 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Request body: {'port': {'device_id': '43046111-0b2a-499f-8353-b8fd77b989ea', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:38.784351 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-78dd2efa-0d23-433c-9028-f55bfe354bda tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:38.813330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e9c02ea-0ad6-4e11-954a-3345721a4518 req-98906a96-6fb6-4fbf-96bd-07eb83f3a61d tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:38.820913 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e9c02ea-0ad6-4e11-954a-3345721a4518 req-98906a96-6fb6-4fbf-96bd-07eb83f3a61d tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 944 time: 0.1309283 May 25 16:03:38.880980 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e9c02ea-0ad6-4e11-954a-3345721a4518 req-e7063dab-2bd5-4e00-9ffd-3ad21be812f0 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0592124 May 25 16:03:38.916052 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-7b680f7d-137e-4299-aa35-935da96a1dc7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:38.917921 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-7b680f7d-137e-4299-aa35-935da96a1dc7 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.1283419 May 25 16:03:38.918510 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Made reservation on behalf of 690d1dab67d340598496a2b626fceaf2 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:38.922023 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:38.971989 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-78dd2efa-0d23-433c-9028-f55bfe354bda tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:38.973922 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-78dd2efa-0d23-433c-9028-f55bfe354bda tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&fields=id HTTP/1.1" status: 200 len: 348 time: 0.4344823 May 25 16:03:39.006380 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-b5732aac-1a5a-4461-968d-c3ddf002a6d9 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0711248 May 25 16:03:39.055917 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:39.148983 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 3935ec55-2a13-43b8-b0bf-f9ef947051dc up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:39.203226 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:39.225232 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-f5d52642-5332-4f00-a333-43046279ca18 service neutron] 10.222.0.82 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2284279 May 25 16:03:39.248164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e2e1ee7-43ec-42b2-a067-859bcfe7cb99 req-a5d70ab7-e0d8-426d-a1d2-0aba9e32f6b8 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:39.249697 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e2e1ee7-43ec-42b2-a067-859bcfe7cb99 req-a5d70ab7-e0d8-426d-a1d2-0aba9e32f6b8 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 931 time: 0.1729729 May 25 16:03:39.399563 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e2e1ee7-43ec-42b2-a067-859bcfe7cb99 req-d15ad9d4-0896-4f3b-9a55-63e135072c6a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1224618 May 25 16:03:39.406713 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 3935ec55-2a13-43b8-b0bf-f9ef947051dc completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:39.413793 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 3935ec55-2a13-43b8-b0bf-f9ef947051dc triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:39.414342 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:39.446117 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-9d38c57b-9fde-4852-ae6b-86263bd18515 service neutron] 10.222.0.82 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2086246 May 25 16:03:39.524419 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-d449a573-bc31-4826-8f53-e12871fb7fa5 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5085058 May 25 16:03:39.736134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:39.737189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Allocated IP 172.20.0.12 (dc1f952c-66ab-4757-a4c9-eef7df57dea8/21c0b655-f786-4f4e-807c-e0313e0d89ae/5df9097f-ef39-4cbc-b791-fce24729f2ed) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:39.791516 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:39.826305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7488d864-7b5b-48fd-9bd7-24c70ff3de42 req-e8f25dea-faec-4f8f-b23e-dfa6cd015ed0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:39.828176 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7488d864-7b5b-48fd-9bd7-24c70ff3de42 req-e8f25dea-faec-4f8f-b23e-dfa6cd015ed0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 945 time: 0.3107972 May 25 16:03:39.887384 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7488d864-7b5b-48fd-9bd7-24c70ff3de42 req-b848a4a0-93f6-45bc-9e39-acbb28e13735 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0482357 May 25 16:03:39.896223 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:39.991211 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:40.002230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 8c1bac9a-e2d6-43eb-9551-100ddcab0607 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:40.124487 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55ddcc70-9e49-4329-95d3-3ead31025574 req-b9953f78-c8f9-4636-8434-a4e3293d2665 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.5925922 May 25 16:03:40.143387 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-6f421d99-c8b3-464d-983c-eabe439768b2 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.005s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:40.315584 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-42893d5e-72a2-4fee-9030-2a471509c357 req-cef8902c-5df6-4a36-98c1-05fabf8be2d3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.2481813 May 25 16:03:40.363711 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-53febcd4-eeaa-431d-872c-024fe30092c5 req-d5f26d27-e2c6-4dff-8b9f-f598106b3ff5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.1807189 May 25 16:03:40.393242 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:40.393242 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.248s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:40.612330 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:40.617999 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:40.618685 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:40.621412 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6926e4cf-09f7-46b0-bfe5-194a94d35df3 req-59fc165b-efa4-4d1d-9fd0-e194f9b2a99e tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:40.623408 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6926e4cf-09f7-46b0-bfe5-194a94d35df3 req-59fc165b-efa4-4d1d-9fd0-e194f9b2a99e tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 944 time: 0.1236358 May 25 16:03:40.730143 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'code': 200} May 25 16:03:40.737335 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6926e4cf-09f7-46b0-bfe5-194a94d35df3 req-880aaf6f-4838-4ee7-86ae-a08d7c9d4ff1 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1051552 May 25 16:03:40.796033 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:40.806176 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:40.867406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-43ef8177-bebf-4ebb-9e3b-efd396d214e2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:40.869404 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-43ef8177-bebf-4ebb-9e3b-efd396d214e2 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1096 time: 0.4995272 May 25 16:03:40.874539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Transition to ACTIVE for port object a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:40.878787 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:40.879894 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:40.929337 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607,segment=NetworkSegment(fcbe700b-26f1-40db-9201-5ec1c3805964),segment_id=fcbe700b-26f1-40db-9201-5ec1c3805964)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:40.976149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95920880-ebb0-4f35-83b4-c7e1ed0f7c45 req-2fa7e11f-242c-47dd-8019-2d5b7dc30a2c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:40.983710 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95920880-ebb0-4f35-83b4-c7e1ed0f7c45 req-2fa7e11f-242c-47dd-8019-2d5b7dc30a2c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 931 time: 0.1233723 May 25 16:03:41.050617 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95920880-ebb0-4f35-83b4-c7e1ed0f7c45 req-5efd26a4-9544-49a5-864d-6ac497bdff61 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0657482 May 25 16:03:41.062936 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.079471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Transition to ACTIVE for port object 5df9097f-ef39-4cbc-b791-fce24729f2ed will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:41.080696 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:41.101862 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.103794 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-003f6b2a-c461-4f5c-97a2-9c22101bc0f7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:41.165205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.195320 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:41.199772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f7bfee22-e180-4278-9032-e84893252bbf req-11610917-da89-4b05-b55d-16d5ccab9cea tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:41.200019 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f7bfee22-e180-4278-9032-e84893252bbf req-11610917-da89-4b05-b55d-16d5ccab9cea tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.0983098 May 25 16:03:41.207925 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:41.218908 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Pushing event updated for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:41.224124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.143s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:41.228478 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-62a48564-b3c3-477a-98d0-ff5aae1de77b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:41.229990 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-62a48564-b3c3-477a-98d0-ff5aae1de77b service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.3465779 May 25 16:03:41.275630 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f7bfee22-e180-4278-9032-e84893252bbf req-21f2ca11-414a-46a5-8a2d-b0845cc42a2d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0686452 May 25 16:03:41.290350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.42', 57092) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:41.384788 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-10dde09b-94c2-4385-97d3-fdc5c45ef42f service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.0918176 May 25 16:03:41.396487 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-2e7fd03b-19d5-4af1-a21f-a1a3b577a4c9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:41.399301 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-2e7fd03b-19d5-4af1-a21f-a1a3b577a4c9 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 1099 time: 0.1076236 May 25 16:03:41.453200 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-13dc7940-1e71-40ed-a5e9-82a885c6a705 req-e919f787-3ac0-4513-9415-de060f939136 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0524352 May 25 16:03:41.471304 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.502463 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:41.502773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Attempting to bind port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:41.503227 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Attempting to bind port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:41.503625 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Attempting to bind port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:41.517402 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:41.517674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Attempting to bind port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 on network feb61d72-e6b4-48c6-bc7c-d61f51987cf9 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:41.529985 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:41.530320 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Checking segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:41.530859 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Bound using segment: {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:41.531244 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Bound port: a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'c781c6c6-6d43-4217-aae6-6935286facf2', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 945, 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:41.532560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.535245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Usage tracker for resource:security_group_rule and tenant:a48c0291f3cf4cdf923cf81e35b969a2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:41.548171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:41.583844 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-ddfd13e6-3af8-48dd-8f44-cfecabf2c760 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:41.585376 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-ddfd13e6-3af8-48dd-8f44-cfecabf2c760 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.1537809 May 25 16:03:41.611818 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:41.646721 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-451add05-e514-46bd-b448-33fdb29328d8 req-064277ff-ab3b-4be3-923b-7e987f15bcda tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.0893621 May 25 16:03:41.770192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.773258 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-927c967a-5a37-43b9-bf32-55b68ed382cd service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1813991 May 25 16:03:41.783334 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.807282 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:41.857584 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 40978) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:41.910228 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:41.926717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:41.954565 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d50106e-25fd-4ac4-9230-3b925fd008da req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.5003123 May 25 16:03:41.972144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Pushing event updated for resources: {'SecurityGroup': ['ID=6f83c3a3-63e7-4308-9897-04fb148f2f37,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:41.975935 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:41.976734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:41.983941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:41.987831 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:41.992679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:41.999001 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:41.999539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:42.009971 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:42.027739 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:42.045677 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:42.051600 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e9a60fe-9728-45d8-afad-5d934f8ba856 req-92afe5fa-1648-400a-bfcd-d2b8210868ee tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:42.059083 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3e9a60fe-9728-45d8-afad-5d934f8ba856 req-92afe5fa-1648-400a-bfcd-d2b8210868ee tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.1899722 May 25 16:03:42.121416 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-a7f3aebf-76db-443e-84bf-b48236a12d69 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3386230 May 25 16:03:42.160360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:42.197032 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 4.1367953 May 25 16:03:42.199669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:42.202806 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3e9a60fe-9728-45d8-afad-5d934f8ba856 req-0a2398fb-40de-41a4-880c-348ab5b6aa5f tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1347878 May 25 16:03:42.221564 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8de62f1a-888c-4a4b-9593-b57bf03a2aed tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:42.346370 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:42.351014 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Pushing event updated for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:42.354028 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2a18df90-4ce0-44d0-8cd0-1d3099bdc21f req-c785e83f-24df-4c9a-8fc6-163258a0e47b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:42.356357 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2a18df90-4ce0-44d0-8cd0-1d3099bdc21f req-c785e83f-24df-4c9a-8fc6-163258a0e47b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 943 time: 0.1388030 May 25 16:03:42.435597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Request body: {'port': {'device_id': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:42.459099 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2a18df90-4ce0-44d0-8cd0-1d3099bdc21f req-549e57e9-b663-4b9e-a756-71858d157a49 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0957494 May 25 16:03:42.490553 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.276s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:42.498257 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Usage tracker for resource:security_group_rule and tenant:d62b64b07ce4451f8f118a5315b2934b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:42.527700 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:42.537403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3fa6f7fb-bd36-4a55-a9a1-399947b05285 None None] Provisioning for port 5df9097f-ef39-4cbc-b791-fce24729f2ed completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:42.543722 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3fa6f7fb-bd36-4a55-a9a1-399947b05285 None None] Provisioning complete for port 5df9097f-ef39-4cbc-b791-fce24729f2ed triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:42.552836 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3fa6f7fb-bd36-4a55-a9a1-399947b05285 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:42.576491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f477c864-ab82-4f04-ba8b-751af398297c req-d7d1ffb0-80f7-43cd-b460-ef0edf3500be tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:42.586597 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f477c864-ab82-4f04-ba8b-751af398297c req-d7d1ffb0-80f7-43cd-b460-ef0edf3500be tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.1314180 May 25 16:03:42.654714 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-3fa6f7fb-bd36-4a55-a9a1-399947b05285 None None] Port 5df9097f-ef39-4cbc-b791-fce24729f2ed cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:42.655466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f477c864-ab82-4f04-ba8b-751af398297c req-87aa1490-5374-42dc-9f2b-b6cb7221fc30 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0678732 May 25 16:03:42.735729 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:42.744804 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d16e3653-2459-4fa7-ad40-58e0bb5ae811 req-9e4662cf-c795-4f14-9fb8-fda9e6057dff service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3997636 May 25 16:03:42.780082 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:42.799676 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0769a9ca-fc4f-4e35-9b1d-e0e1057b6b1d req-ccf43b6f-fae1-4500-9bfc-5d716550e1c9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.1207917 May 25 16:03:42.800540 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:42.801244 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:42.802720 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] 10.222.0.42 "PUT /v2.0/ports/a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 HTTP/1.1" status: 200 len: 1093 time: 4.0536499 May 25 16:03:42.811926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:42.897470 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607', 'code': 200} May 25 16:03:42.992705 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b823bf2-5427-491b-b383-3eec357a81f6 req-06452c9d-81fa-4a53-813b-e5dfbe897183 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:42.998453 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8b823bf2-5427-491b-b383-3eec357a81f6 req-06452c9d-81fa-4a53-813b-e5dfbe897183 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 932 time: 0.1778493 May 25 16:03:43.067268 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:43.109169 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89', 'status': 'completed', 'code': 200} May 25 16:03:43.132720 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8b823bf2-5427-491b-b383-3eec357a81f6 req-1ceefc29-6314-4db6-add0-cd893d549910 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1198068 May 25 16:03:43.173742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 76801a33-2412-4d01-aee4-406f117edbd1 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:43.381091 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:43.455048 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-9c912bcf-0b24-4aeb-b28c-5451d16251a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:43.460331 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-9c912bcf-0b24-4aeb-b28c-5451d16251a4 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 1096 time: 0.3586040 May 25 16:03:43.471322 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:43.584271 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=76801a33-2412-4d01-aee4-406f117edbd1,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:43.628200 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '76801a33-2412-4d01-aee4-406f117edbd1', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'port_id': '76801a33-2412-4d01-aee4-406f117edbd1', 'mac_address': 'fa:16:3e:59:d9:7b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 945, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:43.633480 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d59fbf2e-5aaa-492d-8f7f-2aef6e4ab0b8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:43.789833 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-9e90e4df-1645-4af6-aebf-b8205d17b1b1 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 HTTP/1.1" status: 200 len: 217 time: 0.1642768 May 25 16:03:43.791338 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29b022d1-1bfc-4b6e-b4dc-c506991dddcd req-39b40d42-9bc1-4ca0-8269-e5fbde0264f3 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:43.792929 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29b022d1-1bfc-4b6e-b4dc-c506991dddcd req-39b40d42-9bc1-4ca0-8269-e5fbde0264f3 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.2685630 May 25 16:03:43.796377 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:43.819819 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 76801a33-2412-4d01-aee4-406f117edbd1 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:43.842864 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=76801a33-2412-4d01-aee4-406f117edbd1,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:43.857120 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=d62b64b07ce4451f8f118a5315b2934b&name=default HTTP/1.1" status: 200 len: 2979 time: 1.9754813 May 25 16:03:43.862034 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41032) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:43.864546 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5c46398b-fe66-46ee-a1c1-4830f8db0f3a,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:43.867057 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29b022d1-1bfc-4b6e-b4dc-c506991dddcd req-daaf08aa-2c78-482a-9d73-5c3d98778111 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0555134 May 25 16:03:43.883864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.250s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:43.929987 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-07b229b6-b1d1-44fd-88b4-5351ab5f8f01 req-694aeef5-04b5-418c-bbb4-f9816150cd96 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:43.937113 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-f249da2e-c428-4fe0-b6ac-921f35e005a0 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:43.938015 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-07b229b6-b1d1-44fd-88b4-5351ab5f8f01 req-694aeef5-04b5-418c-bbb4-f9816150cd96 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 943 time: 0.2224729 May 25 16:03:43.938551 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-f249da2e-c428-4fe0-b6ac-921f35e005a0 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.1219304 May 25 16:03:43.947617 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1a8b161-9c46-4cf1-8682-68f393dbe8a0 req-aba4af2d-2d63-4833-a199-ead63901bcce service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:43.949155 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f1a8b161-9c46-4cf1-8682-68f393dbe8a0 req-aba4af2d-2d63-4833-a199-ead63901bcce service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 1098 time: 0.3302670 May 25 16:03:43.977350 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:43.980950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-64f5ec57-1058-42ad-867a-3ccc5f190979 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:44.040459 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-a8097d06-041c-40cf-9b2d-230eb4eb403c service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0891979 May 25 16:03:44.076016 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:44.078119 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-07b229b6-b1d1-44fd-88b4-5351ab5f8f01 req-5ddf68dd-f194-4f45-bb24-6ddc56fb8e50 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1390998 May 25 16:03:44.082947 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.095808 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:44.154979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.155083 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f1a8b161-9c46-4cf1-8682-68f393dbe8a0 req-bee1e64c-ec94-4e23-9258-d9fe89c2f95b service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1316447 May 25 16:03:44.168061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eea46e05-e0db-4a9d-9203-e0e9b29ea8bb req-3ba77b62-8fcf-45ce-bf23-0a83d76654ee tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:44.171273 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eea46e05-e0db-4a9d-9203-e0e9b29ea8bb req-3ba77b62-8fcf-45ce-bf23-0a83d76654ee tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.1774592 May 25 16:03:44.271114 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0492f149-4f85-45d2-bea6-043988577d16 req-f9f4392d-4440-4ec1-a886-3c74163079bb tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0615189 May 25 16:03:44.278678 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['76801a33-2412-4d01-aee4-406f117edbd1'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:44.305349 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eea46e05-e0db-4a9d-9203-e0e9b29ea8bb req-1cdbb5e7-290f-430b-858a-902d4fc38ec5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1330338 May 25 16:03:44.313514 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:44.325528 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.391404 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-d42c1650-af81-4e1e-ae49-15ac7406efeb service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3048441 May 25 16:03:44.551075 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Transition to ACTIVE for port object 5df9097f-ef39-4cbc-b791-fce24729f2ed will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:44.551421 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.558902 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4cbc5cf9-e9d1-41b0-b7c2-3cd16bd0e9a3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:44.574685 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cb9944f3-398f-4105-b28c-bc7efe384192 req-74375987-7017-4752-90fa-e1be4420c538 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1516831 May 25 16:03:44.583206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8641d867-0604-4c83-bb8e-0c0d9807de02 req-776b026d-3fbd-4fed-95a9-bc2c180f8c5b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:44.584650 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8641d867-0604-4c83-bb8e-0c0d9807de02 req-776b026d-3fbd-4fed-95a9-bc2c180f8c5b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 932 time: 0.1687915 May 25 16:03:44.586311 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5c46398b-fe66-46ee-a1c1-4830f8db0f3a,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:44.588845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:44.612569 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.613669 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5c46398b-fe66-46ee-a1c1-4830f8db0f3a HTTP/1.1" status: 204 len: 168 time: 0.7505751 May 25 16:03:44.647532 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8641d867-0604-4c83-bb8e-0c0d9807de02 req-4e64f55e-b81a-4168-9901-8d500000d265 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0547104 May 25 16:03:44.748181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:44.776601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:45.065839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:45.068974 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.292s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:45.114069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Attempting to bind port 5df9097f-ef39-4cbc-b791-fce24729f2ed on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:45.114404 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Attempting to bind port 5df9097f-ef39-4cbc-b791-fce24729f2ed by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:45.115289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Attempting to bind port 5df9097f-ef39-4cbc-b791-fce24729f2ed on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:45.130826 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-9d22670d-3c20-459f-9044-67c03dbb1b96 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.132269 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-9d22670d-3c20-459f-9044-67c03dbb1b96 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 811 time: 0.4639060 May 25 16:03:45.134729 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Port 5df9097f-ef39-4cbc-b791-fce24729f2ed on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:45.135085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Attempting to bind port 5df9097f-ef39-4cbc-b791-fce24729f2ed on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:45.146229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:45.146473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Checking segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:45.146889 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Bound using segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:45.147353 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Bound port: 5df9097f-ef39-4cbc-b791-fce24729f2ed, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:45.147758 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.182501 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 76801a33-2412-4d01-aee4-406f117edbd1 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:45.215277 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-bce11998-4934-4141-8956-4e70cd745a9d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.216797 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-bce11998-4934-4141-8956-4e70cd745a9d service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 1096 time: 0.4061069 May 25 16:03:45.223057 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc0d09d5-d6d8-4b61-877a-0c68542bad93 req-ee9e7927-7f57-4597-84ae-b26e2c3ccd19 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.234535 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bc0d09d5-d6d8-4b61-877a-0c68542bad93 req-ee9e7927-7f57-4597-84ae-b26e2c3ccd19 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.2509241 May 25 16:03:45.327306 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-4cb80bb4-81fb-4dc3-8999-1d53983dcd37 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 380 time: 0.1516302 May 25 16:03:45.387813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'port': {'device_id': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'admin_state_up': True, 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'security_groups': ['e4aeeac6-6179-49e9-a625-03f354d51988']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:45.389197 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bc0d09d5-d6d8-4b61-877a-0c68542bad93 req-ca75d9e8-b2d4-4ccd-82e9-024f1031a80c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1476786 May 25 16:03:45.446832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-887f5f40-919f-4ad8-90f5-2fa879346fd2 req-c8cc9785-8803-484a-b4ab-587b429a6acc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.449435 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-887f5f40-919f-4ad8-90f5-2fa879346fd2 req-c8cc9785-8803-484a-b4ab-587b429a6acc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 943 time: 0.1339273 May 25 16:03:45.463121 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 76801a33-2412-4d01-aee4-406f117edbd1 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:45.470823 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 76801a33-2412-4d01-aee4-406f117edbd1 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:45.471546 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.573686 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-887f5f40-919f-4ad8-90f5-2fa879346fd2 req-3ff45673-dd2b-4adf-9a67-62164066102d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1168435 May 25 16:03:45.595808 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:45.634181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-d95cb09f-7337-4a91-b05f-e2eef05c6bcd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.636468 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-d95cb09f-7337-4a91-b05f-e2eef05c6bcd service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.3873749 May 25 16:03:45.641727 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:45.718715 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-ad8855d3-b75c-4f71-956a-9bc2fa6c1a6f service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 HTTP/1.1" status: 200 len: 217 time: 0.0739284 May 25 16:03:45.719996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:45.742308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-111923bb-89ca-4c1a-bb9d-bf2658460d2a req-36a3cfde-0645-4437-b73f-0ed418912e86 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.751482 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.757066 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-111923bb-89ca-4c1a-bb9d-bf2658460d2a req-36a3cfde-0645-4437-b73f-0ed418912e86 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.1671429 May 25 16:03:45.774401 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41066) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:45.787146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:45.798251 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=76801a33-2412-4d01-aee4-406f117edbd1,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:45.811719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.822364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:45.856048 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-196f1865-0e34-4b45-8071-f86286528b89 req-df894035-50ff-4c79-983d-73d554a8dc71 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0806134 May 25 16:03:45.858072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:45.858945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.902080 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-111923bb-89ca-4c1a-bb9d-bf2658460d2a req-c8d4b398-23a8-4495-8d71-b6c072b61489 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1440105 May 25 16:03:45.903311 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:45.923004 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.923004 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:45.939950 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Usage tracker for resource:security_group_rule and tenant:a8db9364b6444e448872446ec1df5adf is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:45.945704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:45.959857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:45.964111 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-6b4cd93c-6541-4fa0-93d9-1ff807cd9785 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.970079 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-6b4cd93c-6541-4fa0-93d9-1ff807cd9785 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.2197523 May 25 16:03:45.990191 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:45.992961 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:45.996082 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] 10.222.0.42 "PUT /v2.0/ports/5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1094 time: 3.6523952 May 25 16:03:46.014616 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:46.053997 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1', 'code': 200} May 25 16:03:46.112141 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:46.118120 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:46.119307 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Made reservation on behalf of 06dda51337ff4f03a0a16af8eaf39a13 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:03:46.123499 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:46.180270 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'status': 'completed', 'code': 200} May 25 16:03:46.223858 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-cbb03e75-6dcc-44c5-9707-281798f35f19 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.2258556 May 25 16:03:46.269408 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:46.278716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a8c9510-0cb8-42b8-99b8-4fd7678bed4a req-a10de2fd-19de-4744-8a25-5f325031c4c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:46.286219 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a8c9510-0cb8-42b8-99b8-4fd7678bed4a req-a10de2fd-19de-4744-8a25-5f325031c4c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.2755439 May 25 16:03:46.347053 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a8c9510-0cb8-42b8-99b8-4fd7678bed4a req-2c54d6c1-8816-4246-869e-9033ddd548ae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0595653 May 25 16:03:46.359785 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:46.381287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:46.388596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-218c3585-6feb-4118-a4f2-5b4016f320ab admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:46.399365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.541s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:46.435993 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=a8db9364b6444e448872446ec1df5adf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6545489 May 25 16:03:46.435993 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1d7ff424-a712-4d0c-8ef5-cc3bb0c3dd21,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:46.437501 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:46.437979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41084) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:46.551468 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:46.600944 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-fb394ddc-6bd4-4ab8-b559-7d4ede97e522 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3613017 May 25 16:03:46.647755 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:46.733673 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-0fa80aa9-2608-4811-8488-f7075e8a9dd3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:46.735394 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-0fa80aa9-2608-4811-8488-f7075e8a9dd3 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1097 time: 0.5584393 May 25 16:03:46.745461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1f9efcf-a114-447c-85a7-9e5b0cdfa1de req-3648eebd-0737-451a-8627-f2149e0bbc74 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:46.763847 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1f9efcf-a114-447c-85a7-9e5b0cdfa1de req-3648eebd-0737-451a-8627-f2149e0bbc74 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.2646065 May 25 16:03:46.837640 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-3070d494-f50b-4b94-bde4-c619e0b19102 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 217 time: 0.0844214 May 25 16:03:46.839279 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Allocated IP 172.20.0.9 (0105f026-2520-418a-a172-829f4b156413/c52f7a79-f047-4ca9-9f61-86cb7e470fe5/99600d03-17d9-4c12-98bd-6b71a8bbc8e3) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:03:46.876262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f610fa5-df1d-4e0d-80b5-09a6ff09b9a8 req-43264431-bcfb-4c47-90db-6407e96a415c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:46.878992 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5f610fa5-df1d-4e0d-80b5-09a6ff09b9a8 req-43264431-bcfb-4c47-90db-6407e96a415c tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.3285172 May 25 16:03:46.964308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-d1f9efcf-a114-447c-85a7-9e5b0cdfa1de req-d7cab529-0549-4d4e-8cb6-e7f01f1c5c66 service neutron] Synchronizing usage tracker for tenant:a48c0291f3cf4cdf923cf81e35b969a2 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:03:46.970074 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-7fcb7473-ff1f-46b7-8ef1-96e44e4e8b11 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:46.971560 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-7fcb7473-ff1f-46b7-8ef1-96e44e4e8b11 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.1281469 May 25 16:03:46.973305 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-ae1b9d3b-8863-4bf3-ab26-74327c963db5 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3360000 May 25 16:03:46.998185 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5f610fa5-df1d-4e0d-80b5-09a6ff09b9a8 req-14e13007-c4c4-4030-9808-c3ec05105ce9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0993381 May 25 16:03:47.042852 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2cfd552-000f-4187-b68c-dcc9f0ecb53f req-ca5fe0a1-2d7e-4afe-8236-7825facae51d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:47.044575 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e2cfd552-000f-4187-b68c-dcc9f0ecb53f req-ca5fe0a1-2d7e-4afe-8236-7825facae51d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 943 time: 0.2748225 May 25 16:03:47.069872 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.077870 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:47.094380 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1d7ff424-a712-4d0c-8ef5-cc3bb0c3dd21,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:47.097374 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:47.106096 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.111249 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/1d7ff424-a712-4d0c-8ef5-cc3bb0c3dd21 HTTP/1.1" status: 204 len: 168 time: 0.6687875 May 25 16:03:47.111249 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-d1f9efcf-a114-447c-85a7-9e5b0cdfa1de req-d7cab529-0549-4d4e-8cb6-e7f01f1c5c66 service neutron] Unset dirty status for tenant:a48c0291f3cf4cdf923cf81e35b969a2 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:03:47.111249 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1f9efcf-a114-447c-85a7-9e5b0cdfa1de req-d7cab529-0549-4d4e-8cb6-e7f01f1c5c66 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3393352 May 25 16:03:47.141685 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e2cfd552-000f-4187-b68c-dcc9f0ecb53f req-1e6b6ea6-ff17-4417-845e-88eb4fcc13dc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0903044 May 25 16:03:47.193897 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:47.238431 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.259501 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-457c2e8d-d80d-4625-aa94-f6269c96e6a8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:47.261178 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-457c2e8d-d80d-4625-aa94-f6269c96e6a8 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2627714 May 25 16:03:47.272158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Transition to ACTIVE for port object 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:47.316746 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-207a9d91-512e-4414-b031-8a848442836b req-3d6ccd05-caa5-44ac-8183-83d5fcbcb710 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.1272323 May 25 16:03:47.505658 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:47.523402 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89', 'network_id': 'feb61d72-e6b4-48c6-bc7c-d61f51987cf9', 'port_id': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89', 'mac_address': 'fa:16:3e:20:34:79', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 945, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '051377f2-6b21-4071-b8f5-21107e3c870f', 'ip_address': '172.20.0.2'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:47.588946 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-fcf02e31-db74-4b8c-9bc9-236ff5eee9e6 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2026446 May 25 16:03:47.590557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.600219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.607010 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:47.626214 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a5e90c8-e91d-4758-bcde-c4bff21ced12 req-00099804-e316-4e7c-8206-80527618075b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:47.632088 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:47.636708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a5e90c8-e91d-4758-bcde-c4bff21ced12 req-00099804-e316-4e7c-8206-80527618075b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1236465 May 25 16:03:47.659655 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:47.695833 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Network 0105f026-2520-418a-a172-829f4b156413 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:03:47.704952 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a5e90c8-e91d-4758-bcde-c4bff21ced12 req-3ab00365-e42f-4938-976f-cfaeb7c93a6f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0673683 May 25 16:03:47.762426 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:03:47.773711 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:03:47.779376 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:47.780028 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.795517 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:47.798471 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ff89a671-25a4-4148-98f0-e0c73b1c83fe None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:47.829770 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:47.835271 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.228s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:47.891318 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7af903ba-07ed-4170-a610-3d5505d7d3df req-212bb8c7-51e1-4a64-b7d6-74aa3d012197 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2420349 May 25 16:03:47.934355 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:47.939898 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.141s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:48.005415 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 2.6223738 May 25 16:03:48.035463 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Request body: {'port': {'device_id': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:03:48.193277 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:48.202104 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['a7ff877f-0a9b-46af-9cc4-578eeaaa2e89'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:48.349288 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-9e986df3-7970-4b09-8038-e7d03b000029 None None] Provisioning for port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:48.364536 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-9e986df3-7970-4b09-8038-e7d03b000029 None None] Provisioning complete for port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:48.365426 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9e986df3-7970-4b09-8038-e7d03b000029 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:48.533843 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-9e986df3-7970-4b09-8038-e7d03b000029 None None] Port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:03:48.566592 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32377e36-c963-4745-8604-bb7a5e453d0f req-60224ad7-7a3f-4c36-9878-3b1b747354ee service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:48.567948 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c382d5a3-d810-4299-9fb3-1cc5547d70f7 req-e1de1aee-6aca-447a-818b-18bd5d2f2451 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:48.573938 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c382d5a3-d810-4299-9fb3-1cc5547d70f7 req-e1de1aee-6aca-447a-818b-18bd5d2f2451 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 944 time: 0.2360997 May 25 16:03:48.577750 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-60224ad7-7a3f-4c36-9878-3b1b747354ee service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1097 time: 0.2792318 May 25 16:03:48.681258 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c382d5a3-d810-4299-9fb3-1cc5547d70f7 req-e5263cff-fe4d-4cf8-a0c7-80d760758c2d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1013618 May 25 16:03:48.726430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55f95cb0-467a-418c-8b7c-12344b1f2d23 req-f5a2320b-2d30-491b-b332-8fb704c23d92 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:48.736829 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55f95cb0-467a-418c-8b7c-12344b1f2d23 req-f5a2320b-2d30-491b-b332-8fb704c23d92 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 931 time: 0.1308863 May 25 16:03:48.808642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55f95cb0-467a-418c-8b7c-12344b1f2d23 req-1ca837e2-5b9b-4ad0-ac65-29e0db9f1376 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0681303 May 25 16:03:48.950208 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41148) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:48.974410 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2dca9643-27fe-459c-b9cb-3d7a2f560d24 req-8f641e12-e808-4227-a68b-5ce3a3ece0c1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:48.983010 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2dca9643-27fe-459c-b9cb-3d7a2f560d24 req-8f641e12-e808-4227-a68b-5ce3a3ece0c1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1364503 May 25 16:03:49.042170 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2dca9643-27fe-459c-b9cb-3d7a2f560d24 req-3bc77933-a35e-4452-be92-36d3871ba0f6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0576627 May 25 16:03:49.057782 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32377e36-c963-4745-8604-bb7a5e453d0f req-4ed9ea82-f55b-4c6d-be96-9931a942be94 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:49.059243 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-4ed9ea82-f55b-4c6d-be96-9931a942be94 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.4736459 May 25 16:03:49.125330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.145248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:49.149375 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Usage tracker for resource:security_group_rule and tenant:39d9bed61ea04645b4ca499d85c9abeb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:49.153858 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:49.160251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:49.261421 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-2fd80dca-2e73-43c6-9586-425fcfed837a service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 217 time: 0.1653645 May 25 16:03:49.280189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.311960 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:49.338249 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.358958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.370298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Transition to ACTIVE for port object 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:03:49.372482 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:49.374149 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32377e36-c963-4745-8604-bb7a5e453d0f req-9dd9b795-fa19-4a00-9c07-96772e5acd74 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:49.375701 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-9dd9b795-fa19-4a00-9c07-96772e5acd74 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.1075556 May 25 16:03:49.418691 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=39d9bed61ea04645b4ca499d85c9abeb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4674456 May 25 16:03:49.419654 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=3bf99632-7ab8-4def-9b11-df7e6946dc8e,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:49.422943 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:49.423526 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41164) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:49.464267 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.473506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:49.490848 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.521883 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.600340 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32377e36-c963-4745-8604-bb7a5e453d0f req-c7179e81-f99f-44f8-b00f-499c53309a9d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:49.601835 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-c7179e81-f99f-44f8-b00f-499c53309a9d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2169969 May 25 16:03:49.665564 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Pushing event updated for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:49.677143 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.204s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:49.702786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.710112 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-708a83af-428d-488c-8ce7-17e51a29ff0d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:49.735379 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=3bf99632-7ab8-4def-9b11-df7e6946dc8e,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:49.738153 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:49.755963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.759624 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/3bf99632-7ab8-4def-9b11-df7e6946dc8e HTTP/1.1" status: 204 len: 168 time: 0.3326705 May 25 16:03:49.796636 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Attempting to bind port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:03:49.796887 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Attempting to bind port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:03:49.802836 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Attempting to bind port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:49.818044 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 on network 0105f026-2520-418a-a172-829f4b156413 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:03:49.818319 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Attempting to bind port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:03:49.836028 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 3, 39), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:03:49.836370 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Checking segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:03:49.836805 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Bound using segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:03:49.837255 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Bound port: 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:03:49.837601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:49.986034 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-6297ec34-b38d-4084-b790-3e4e411a2b1a service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3740993 May 25 16:03:49.997836 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-797f64c7-2dc3-4298-a5a6-43caa43c1d57 req-117de9f2-c2e5-4310-ae87-c36a44eb3b79 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:50.003527 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-797f64c7-2dc3-4298-a5a6-43caa43c1d57 req-117de9f2-c2e5-4310-ae87-c36a44eb3b79 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 944 time: 0.1367292 May 25 16:03:50.055722 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-797f64c7-2dc3-4298-a5a6-43caa43c1d57 req-6729fc3c-2d04-4e1c-947f-bcf13045fd88 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0488577 May 25 16:03:50.119850 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:03:50.133172 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:03:50.168497 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-32377e36-c963-4745-8604-bb7a5e453d0f req-93301684-38be-4b45-972f-638d2827f4b9 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1763108 May 25 16:03:50.248039 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-357ef99a-f181-41d4-8e43-f19b346358e9 req-087557c3-12ed-4d77-94f2-415ca60455dd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:50.249502 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-357ef99a-f181-41d4-8e43-f19b346358e9 req-087557c3-12ed-4d77-94f2-415ca60455dd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.0894358 May 25 16:03:50.252452 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:50.262307 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:50.333369 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:50.333930 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:50.335353 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] 10.222.0.42 "PUT /v2.0/ports/99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 1093 time: 2.3045728 May 25 16:03:50.335983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:50.367851 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Pushing event updated for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:50.372427 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:50.380197 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-357ef99a-f181-41d4-8e43-f19b346358e9 req-5acca920-ef5b-4888-8f3e-4df53bba5f97 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1174815 May 25 16:03:50.712015 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-36e94031-cbcd-407b-8e3f-3f6607c8a61a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:50.712015 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-36e94031-cbcd-407b-8e3f-3f6607c8a61a service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1096 time: 0.1972764 May 25 16:03:50.794138 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-5d9cf105-b88e-40c3-9186-766218f1e0a1 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 217 time: 0.0693548 May 25 16:03:50.928632 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41220) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:50.963795 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:50.993773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-ef95d67f-5c2c-4280-84e2-a5179925e2ab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:50.999336 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-ef95d67f-5c2c-4280-84e2-a5179925e2ab service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.1992028 May 25 16:03:51.002089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:51.043244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Usage tracker for resource:security_group_rule and tenant:2196a6f414114d229e2875f4f72b31ce is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:51.043244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:51.052271 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:51.100157 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-b2cf4b97-88c9-4677-892f-9f5889bef73d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:51.102023 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-b2cf4b97-88c9-4677-892f-9f5889bef73d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0851867 May 25 16:03:51.140776 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68508259-3565-4c8f-b6dc-5c2bace3ebe7 req-3f27eb49-895b-4d11-9dd5-ff7263e802d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:51.148320 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68508259-3565-4c8f-b6dc-5c2bace3ebe7 req-3f27eb49-895b-4d11-9dd5-ff7263e802d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1473818 May 25 16:03:51.219370 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:51.228198 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:51.232264 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.273329 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68508259-3565-4c8f-b6dc-5c2bace3ebe7 req-32c720ee-c970-4e9f-abb5-d2e8481b45f3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1238914 May 25 16:03:51.327501 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-ba26630e-f66f-4bb6-9f9a-78060a1bf83d service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2141926 May 25 16:03:51.384137 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'status': 'completed', 'code': 200} May 25 16:03:51.501851 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.518200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c1feb05f-4537-4198-9386-b3865996d792 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:51.519824 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:51.540889 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:51.575789 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-caa7a5a7-278e-4072-b0a1-8dbf25bd4691 req-ea96327d-8e5c-44bd-8499-0115004760b0 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2105784 May 25 16:03:51.584797 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=70e6887a-2cf2-4fb6-b4a7-683ac0af7c45,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:51.590751 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=2196a6f414114d229e2875f4f72b31ce&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6601799 May 25 16:03:51.592076 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:51.593486 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41240) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:51.627916 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.628699 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8ee0d8b-9a38-4ca2-a0a6-b19f4ac891c8 req-9a8309fa-5ec3-4a25-bf08-ebe4db1ca58e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:51.628699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8ee0d8b-9a38-4ca2-a0a6-b19f4ac891c8 req-9a8309fa-5ec3-4a25-bf08-ebe4db1ca58e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 944 time: 0.1653812 May 25 16:03:51.653118 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:51.673429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:51.675193 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:51.677930 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8ee0d8b-9a38-4ca2-a0a6-b19f4ac891c8 req-b29aa71f-63b8-4b8c-82a5-189f85a3e498 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0439014 May 25 16:03:51.683485 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:51.683912 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.689934 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.727124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.858953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.870989 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:51.876230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:51.970303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-642d05e7-5942-4223-b189-51c9b102406d req-0958eea6-41de-40bc-a008-18cfc1536af3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:51.972392 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-0958eea6-41de-40bc-a008-18cfc1536af3 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1096 time: 0.1206510 May 25 16:03:51.976290 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:51.990505 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d092930-bac1-4c32-84b3-ea6a7cf3ba98 req-2e7480f3-ab54-4a94-a748-2934e60a3d6e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:51.995533 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d092930-bac1-4c32-84b3-ea6a7cf3ba98 req-2e7480f3-ab54-4a94-a748-2934e60a3d6e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.1919894 May 25 16:03:51.995670 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:52.007039 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:52.018702 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:52.068288 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.073982 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d092930-bac1-4c32-84b3-ea6a7cf3ba98 req-273ef1fb-9c70-438a-bcdc-d0d00937201f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0748758 May 25 16:03:52.085204 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4a951a15-50f7-4985-9eac-f25b491a4b29 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:52.087269 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.096922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:52.107806 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=70e6887a-2cf2-4fb6-b4a7-683ac0af7c45,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:52.112813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:52.113537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.114695 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/70e6887a-2cf2-4fb6-b4a7-683ac0af7c45 HTTP/1.1" status: 204 len: 168 time: 0.5202055 May 25 16:03:52.199150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-642d05e7-5942-4223-b189-51c9b102406d req-053fdb82-7955-4081-8953-462c630adbb6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:52.200688 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-053fdb82-7955-4081-8953-462c630adbb6 service neutron] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 912 time: 0.2202430 May 25 16:03:52.267503 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89', 'code': 200} May 25 16:03:52.346744 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:52.351382 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.257s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:52.405798 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41264) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:52.541574 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c9e0f2b-508c-4c86-a40a-8cc964654ebc req-1de26a20-7756-4377-9cb8-9ddfba8e434f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:52.543820 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2c9e0f2b-508c-4c86-a40a-8cc964654ebc req-1de26a20-7756-4377-9cb8-9ddfba8e434f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1184368 May 25 16:03:52.648789 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2c9e0f2b-508c-4c86-a40a-8cc964654ebc req-3a32fa84-589c-4780-a0c0-215d2fb34c07 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0930810 May 25 16:03:52.751768 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.772035 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:52.778675 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Usage tracker for resource:security_group_rule and tenant:43b0524f52d845a09597307d390aa7ff is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:52.781933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:52.788506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:52.897429 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.905678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e2aa77b-e8d3-4182-9641-306207e9d1ef req-7353a35d-63ec-4eec-b582-126e2f5e9942 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:52.907316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e2aa77b-e8d3-4182-9641-306207e9d1ef req-7353a35d-63ec-4eec-b582-126e2f5e9942 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1053300 May 25 16:03:52.965742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:52.969175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e2aa77b-e8d3-4182-9641-306207e9d1ef req-7769ed66-a3bc-4738-b731-3f721189329c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0546999 May 25 16:03:52.974344 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:53.018166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5c51f80a-afe0-4e32-b193-c6da6b16bd34,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:53.023024 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=43b0524f52d845a09597307d390aa7ff&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6160710 May 25 16:03:53.024042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:53.028521 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41284) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:53.111532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:53.148015 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:53.372531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6be6a12-ac7a-4521-af6b-a358bb5de21e req-bc3ff71c-1932-4f90-8d3d-0bdb5e74da9e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:53.375697 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d6be6a12-ac7a-4521-af6b-a358bb5de21e req-bc3ff71c-1932-4f90-8d3d-0bdb5e74da9e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.1391058 May 25 16:03:53.378022 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:53.395709 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:53.428104 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5c51f80a-afe0-4e32-b193-c6da6b16bd34,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:53.431633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:53.436269 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5c51f80a-afe0-4e32-b193-c6da6b16bd34 HTTP/1.1" status: 204 len: 168 time: 0.4041963 May 25 16:03:53.436269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:53.464990 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d6be6a12-ac7a-4521-af6b-a358bb5de21e req-8f49ea13-2ba0-40b5-82f1-6b14b2314b33 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0792897 May 25 16:03:53.869060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4389cd79-c01f-416a-8f67-83f8cc2740bc req-4b5963a6-b193-4a17-b9e9-ad1a9bb872ac tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:53.869060 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4389cd79-c01f-416a-8f67-83f8cc2740bc req-4b5963a6-b193-4a17-b9e9-ad1a9bb872ac tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.0946743 May 25 16:03:53.918331 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4389cd79-c01f-416a-8f67-83f8cc2740bc req-dfddd755-bfca-42bd-aec3-940a26041b80 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0434709 May 25 16:03:54.184051 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-98337dea-5536-4a71-a58f-fe06b659e06c req-8729cc3b-e3d2-4a8c-b8ba-4c4f8fe8bfbc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:54.185587 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-98337dea-5536-4a71-a58f-fe06b659e06c req-8729cc3b-e3d2-4a8c-b8ba-4c4f8fe8bfbc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0920279 May 25 16:03:54.236643 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-98337dea-5536-4a71-a58f-fe06b659e06c req-d1cce693-b028-481b-b71e-3956d7566897 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0448506 May 25 16:03:54.243030 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41322) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:54.265319 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.282128 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:03:54.287488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Usage tracker for resource:security_group_rule and tenant:075bbc9fc4de4b61b024a685df5a589c is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:03:54.291079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:03:54.299179 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:03:54.401512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.542791 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.554705 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:54.624455 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=075bbc9fc4de4b61b024a685df5a589c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3797507 May 25 16:03:54.625235 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=decba810-6769-429d-b1eb-080a341a5574,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:54.628819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41336) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:03:54.632021 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:54.712759 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.751904 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.811149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d8b6719-952b-452d-9e41-5cd6033c60fa req-3ae70ba3-af87-43a1-b2d7-b651c82ed636 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:54.813764 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d8b6719-952b-452d-9e41-5cd6033c60fa req-3ae70ba3-af87-43a1-b2d7-b651c82ed636 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.1505260 May 25 16:03:54.876232 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d8b6719-952b-452d-9e41-5cd6033c60fa req-5ebc9918-9493-4631-9788-e4cc19ed6ff9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0559115 May 25 16:03:54.920072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.924755 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:54.936953 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=decba810-6769-429d-b1eb-080a341a5574,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:54.941179 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:54.945573 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:54.946902 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/decba810-6769-429d-b1eb-080a341a5574 HTTP/1.1" status: 204 len: 168 time: 0.3168035 May 25 16:03:55.134192 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-11a77c19-1f60-4c46-b567-fb091c7234d8 req-5a7e47a7-6ff7-42dd-af51-160635ffd91a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:55.136351 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-11a77c19-1f60-4c46-b567-fb091c7234d8 req-5a7e47a7-6ff7-42dd-af51-160635ffd91a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.0866263 May 25 16:03:55.187679 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-683c969b-c567-46be-96d4-999a208d120d service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 217 time: 0.0729768 May 25 16:03:55.200472 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-11a77c19-1f60-4c46-b567-fb091c7234d8 req-a0048fee-e867-45b7-8b66-c8aa21ba336e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0580301 May 25 16:03:55.215204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['5df9097f-ef39-4cbc-b791-fce24729f2ed'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:55.280809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-642d05e7-5942-4223-b189-51c9b102406d req-0a212131-482b-4665-9af7-29a02a3ca3a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:55.282540 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-0a212131-482b-4665-9af7-29a02a3ca3a9 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.0725322 May 25 16:03:55.378476 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-642d05e7-5942-4223-b189-51c9b102406d req-37b30d29-03d1-4ed6-95b3-657f039506d7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:55.380410 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-37b30d29-03d1-4ed6-95b3-657f039506d7 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0893762 May 25 16:03:55.468773 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80551eff-6472-4d98-a87c-01fd4d6fa5a7 req-c76fb0d8-bb44-4c87-b77d-a89ffe518c84 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:55.470230 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-80551eff-6472-4d98-a87c-01fd4d6fa5a7 req-c76fb0d8-bb44-4c87-b77d-a89ffe518c84 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1008489 May 25 16:03:55.516168 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-80551eff-6472-4d98-a87c-01fd4d6fa5a7 req-ac8cc097-b93f-4ef8-9f95-f311f6a90f4f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399983 May 25 16:03:55.552614 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-54149f0d-b1a2-47e6-9502-1410e04831f0 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1533425 May 25 16:03:55.778769 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-642d05e7-5942-4223-b189-51c9b102406d req-866d6380-3bf9-47c0-ad45-8671b5bfec64 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2071228 May 25 16:03:55.781800 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a3a2632-7cc6-432f-b7fa-d9973534ffcf req-5d9b09c8-8807-4d28-9a7b-9a1c4ea429e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:55.783349 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5a3a2632-7cc6-432f-b7fa-d9973534ffcf req-5d9b09c8-8807-4d28-9a7b-9a1c4ea429e7 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 1098 time: 0.1922853 May 25 16:03:55.856561 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5a3a2632-7cc6-432f-b7fa-d9973534ffcf req-bd9fde21-af22-4fa1-b80e-64a7f2691fb7 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0442653 May 25 16:03:56.106394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd1f6657-ea33-4af1-bfbc-c68b3f795084 req-ca0a3642-8bbe-47a9-81bc-f3b6d43e69e3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:56.108059 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd1f6657-ea33-4af1-bfbc-c68b3f795084 req-ca0a3642-8bbe-47a9-81bc-f3b6d43e69e3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.0995462 May 25 16:03:56.173022 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd1f6657-ea33-4af1-bfbc-c68b3f795084 req-9d8ee7d8-4779-44a6-976e-f0b15edbb4e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0585177 May 25 16:03:56.428668 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4677bccc-ac18-4121-9634-fcd7b8a851b6 req-7b66545f-6ab9-4a4a-93c2-820d79cbc18b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:56.430153 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4677bccc-ac18-4121-9634-fcd7b8a851b6 req-7b66545f-6ab9-4a4a-93c2-820d79cbc18b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.0982308 May 25 16:03:56.480826 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4677bccc-ac18-4121-9634-fcd7b8a851b6 req-cb2dfdab-0a51-4f6e-8806-cc548138d4d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0439684 May 25 16:03:56.726892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f7687b4-2e0a-4366-b6fe-cafebc96c937 req-5854a1e0-4ad0-4e9c-8381-668115fdbfe6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:56.728424 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f7687b4-2e0a-4366-b6fe-cafebc96c937 req-5854a1e0-4ad0-4e9c-8381-668115fdbfe6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0995188 May 25 16:03:56.792068 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f7687b4-2e0a-4366-b6fe-cafebc96c937 req-746bc1bd-4637-4259-a1aa-8e411df35dd5 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0564165 May 25 16:03:57.116750 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:03:57.124885 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:03:57.434270 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:57.460384 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'port_id': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'mac_address': 'fa:16:3e:19:6a:7a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 932, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:03:57.494792 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fac3de2d-ac87-4751-a329-fc8ee94c7765 req-92faf44a-4428-45eb-9e43-b69e664a35e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:57.496552 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fac3de2d-ac87-4751-a329-fc8ee94c7765 req-92faf44a-4428-45eb-9e43-b69e664a35e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 943 time: 0.1069605 May 25 16:03:57.538712 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:57.551571 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fac3de2d-ac87-4751-a329-fc8ee94c7765 req-b3e9a596-9d4e-4864-bc84-e06ae8b75509 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0486422 May 25 16:03:57.574835 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:03:57.594963 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:57.719133 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53527b54-0f53-46d4-9f45-61a2ba9d58ba req-7e6288ce-c922-486b-ad5a-1ca827cc27bb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:57.720540 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-53527b54-0f53-46d4-9f45-61a2ba9d58ba req-7e6288ce-c922-486b-ad5a-1ca827cc27bb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.0842009 May 25 16:03:57.768898 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-53527b54-0f53-46d4-9f45-61a2ba9d58ba req-15bb7d9c-0ee2-4b98-a2ef-76d60f27702a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0415194 May 25 16:03:57.803867 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:57.803867 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:57.845135 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 5df9097f-ef39-4cbc-b791-fce24729f2ed completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:03:57.874628 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 5df9097f-ef39-4cbc-b791-fce24729f2ed triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:03:57.875497 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:57.946777 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:57.973976 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:58.062567 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-887111e4-bc9b-4c72-9fd3-9288c70b3922 req-f1e2cb30-abaa-49bb-bce9-953be8c976b1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:58.064540 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-887111e4-bc9b-4c72-9fd3-9288c70b3922 req-f1e2cb30-abaa-49bb-bce9-953be8c976b1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1333323 May 25 16:03:58.117498 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:58.154622 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:03:58.178894 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:03:58.216005 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'code': 200} May 25 16:03:58.234174 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-887111e4-bc9b-4c72-9fd3-9288c70b3922 req-b83f7b11-1a4f-4a75-926c-06b1cc5b4d73 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1628158 May 25 16:03:58.289278 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['99600d03-17d9-4c12-98bd-6b71a8bbc8e3'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:03:58.423507 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:03:58.430893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:03:58.563927 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:03:58.569715 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:03:58.774751 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b620959e-b8a0-444c-969e-86dcfe3802bc req-ff5cb6c3-05a9-45aa-9690-f21a7f01349e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:58.774751 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b620959e-b8a0-444c-969e-86dcfe3802bc req-ff5cb6c3-05a9-45aa-9690-f21a7f01349e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 944 time: 0.0943718 May 25 16:03:58.854425 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b620959e-b8a0-444c-969e-86dcfe3802bc req-2ea60200-9c0e-4190-b94e-3bbd8e7c86ea tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0722246 May 25 16:03:58.874611 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-62026a7f-6d4e-43d6-a96a-f4d380070e53 req-7dccee16-57a4-4cee-9843-bcca6ec9180c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:58.877652 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-62026a7f-6d4e-43d6-a96a-f4d380070e53 req-7dccee16-57a4-4cee-9843-bcca6ec9180c service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1099 time: 0.1292353 May 25 16:03:58.961830 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-62026a7f-6d4e-43d6-a96a-f4d380070e53 req-8ef07e39-5ad9-4aca-b8c3-d398257ae056 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0752437 May 25 16:03:59.006423 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e113571-586a-43fe-a908-c9dd243cee9b req-dc576f85-35ac-49e7-b964-f3a6913cb0f4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:59.007807 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e113571-586a-43fe-a908-c9dd243cee9b req-dc576f85-35ac-49e7-b964-f3a6913cb0f4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.0954413 May 25 16:03:59.058754 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e113571-586a-43fe-a908-c9dd243cee9b req-b17ad995-375a-4ab8-b05f-35b800d756a9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0443380 May 25 16:03:59.440869 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47d23290-3833-478b-832e-193802c47a2a req-19078350-57d5-43b1-a3f9-5e39342a89a6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:03:59.442322 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-47d23290-3833-478b-832e-193802c47a2a req-19078350-57d5-43b1-a3f9-5e39342a89a6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0984414 May 25 16:03:59.489839 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-47d23290-3833-478b-832e-193802c47a2a req-0fa5e836-37c6-4aa9-ae07-86df1716192e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0410423 May 25 16:04:00.109355 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-19ef1ea0-f93b-4921-8042-3de2c6b91d29 req-760a6e1d-bd38-4682-ad85-48a2b1c94fd3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.112024 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-19ef1ea0-f93b-4921-8042-3de2c6b91d29 req-760a6e1d-bd38-4682-ad85-48a2b1c94fd3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 944 time: 0.1231792 May 25 16:04:00.168079 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-19ef1ea0-f93b-4921-8042-3de2c6b91d29 req-f18c3200-020b-4f10-8aeb-7ac33088e071 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0490568 May 25 16:04:00.393456 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-bf92b777-e35d-4028-b66b-3156f513f749 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.395539 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-bf92b777-e35d-4028-b66b-3156f513f749 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 1098 time: 0.1444695 May 25 16:04:00.437887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edc6d424-94ef-4245-9b3e-e3c93d1ae759 req-f27ee0b4-1341-40c8-8597-7bab1761dbf2 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.452531 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-edc6d424-94ef-4245-9b3e-e3c93d1ae759 req-f27ee0b4-1341-40c8-8597-7bab1761dbf2 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.2537236 May 25 16:04:00.533482 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d87c0de3-1f2d-4b6d-b62d-07770e3f45c9 req-2b70c3cd-cc15-481f-855c-ff38d573a37f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.536170 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d87c0de3-1f2d-4b6d-b62d-07770e3f45c9 req-2b70c3cd-cc15-481f-855c-ff38d573a37f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2424543 May 25 16:04:00.599894 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-edc6d424-94ef-4245-9b3e-e3c93d1ae759 req-8e5bb709-7e4b-475e-8322-3dab69b69cd9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1459031 May 25 16:04:00.674398 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d87c0de3-1f2d-4b6d-b62d-07770e3f45c9 req-a78f8498-14d9-47bc-8d52-0736edce75e2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1315439 May 25 16:04:00.687345 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-49d92eda-c95c-4d26-9c0b-a44d179ab6b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.687345 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-49d92eda-c95c-4d26-9c0b-a44d179ab6b6 service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.2787070 May 25 16:04:00.871444 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-1b53fc87-f081-4c6b-aac8-6c3d420938ec service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 HTTP/1.1" status: 200 len: 217 time: 0.1640804 May 25 16:04:00.942895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-f759af83-3ca3-40c3-8125-d1ce29c1bd0b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:00.944350 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-f759af83-3ca3-40c3-8125-d1ce29c1bd0b service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.0608830 May 25 16:04:01.039609 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-2d47b1f2-a2cd-48e0-9aef-d31d3d7b4234 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0504599 May 25 16:04:01.116101 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:04:01.257481 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-c5ed01f9-874c-4327-a53e-5662d0ae371a service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2073760 May 25 16:04:01.418573 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc884abf-72b8-4170-84c9-08988316444b req-53550c9f-50a6-4778-86bf-0140d99ce52e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:01.420175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc884abf-72b8-4170-84c9-08988316444b req-53550c9f-50a6-4778-86bf-0140d99ce52e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 944 time: 0.0984838 May 25 16:04:01.428685 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:04:01.434674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:04:01.439205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:01.491249 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc884abf-72b8-4170-84c9-08988316444b req-5d69b858-5887-4159-989c-dbe975f8165f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0588422 May 25 16:04:01.616262 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:01.652076 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:01.667155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:01.717650 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'code': 200} May 25 16:04:01.770167 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:01.777902 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:01.862462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c628a93-22ca-4e32-bf88-9d2a3859b8da req-707bb5ce-ec4f-4596-b934-ea52755c64dd tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:01.862732 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2c628a93-22ca-4e32-bf88-9d2a3859b8da req-707bb5ce-ec4f-4596-b934-ea52755c64dd tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1374171 May 25 16:04:01.895221 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:01.905878 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.906225 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.906557 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.907082 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.907235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.907741 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.907889 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.908247 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.908742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.909113 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.909389 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.909709 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:01.914096 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:01.971842 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2c628a93-22ca-4e32-bf88-9d2a3859b8da req-2703e7bd-bb8f-4315-8dd7-657b03ac5dab tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1011429 May 25 16:04:02.014360 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1cd8603-51ed-4fab-8621-419129678e92 req-e9f40dd8-a9fa-482f-b7aa-0004fcadfaae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:02.016089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1cd8603-51ed-4fab-8621-419129678e92 req-e9f40dd8-a9fa-482f-b7aa-0004fcadfaae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1210113 May 25 16:04:02.067622 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1cd8603-51ed-4fab-8621-419129678e92 req-a8f55d25-76f1-4da1-a260-2db478bef32b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0447519 May 25 16:04:02.278991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1e8a994-7ca0-4d85-a8f0-fd342ae862e5 req-79f8b529-bf37-48a8-a305-eb8f5223d2dd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:02.281666 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c1e8a994-7ca0-4d85-a8f0-fd342ae862e5 req-79f8b529-bf37-48a8-a305-eb8f5223d2dd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.0887091 May 25 16:04:02.289375 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41524) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:02.295748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'floatingip': {'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:02.393843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:03.005329 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d414425c-2749-46df-97f5-0963db7cb634 req-0d4e227b-21cb-493b-9b8b-c6be4229dd2d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:03.006950 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d414425c-2749-46df-97f5-0963db7cb634 req-0d4e227b-21cb-493b-9b8b-c6be4229dd2d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.3723750 May 25 16:04:03.504898 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d414425c-2749-46df-97f5-0963db7cb634 req-d75baab5-3277-4ea4-b07c-b09ea7415ace tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.4717662 May 25 16:04:03.672726 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-914819d2-672c-4bb7-8424-017e1b7d32ac req-b167e6b4-0ea7-48f8-ad05-7ba0f53cdab9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:03.675376 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-914819d2-672c-4bb7-8424-017e1b7d32ac req-b167e6b4-0ea7-48f8-ad05-7ba0f53cdab9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.4388852 May 25 16:04:03.678279 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.db.resource_extend [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] It took 0.13 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 25 16:04:03.928252 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-914819d2-672c-4bb7-8424-017e1b7d32ac req-b9cebd8e-3091-4132-91f0-7af6e360186d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1413696 May 25 16:04:03.932129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:04.000230 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:04.187755 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Floating IP 36261b1e-35fc-430b-8736-29cd641cfe62 associated. External IP: 172.27.2.163, port: 5df9097f-ef39-4cbc-b791-fce24729f2ed. May 25 16:04:04.607965 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3569f4b2-c5c4-4ae3-ac52-35e90eeb484c req-51cdcc6c-8cf6-4b5b-bcb3-60b627db95b6 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2016621 May 25 16:04:04.659387 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:04:05.021821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08e4063a-59d1-409a-9507-7b5356a1a2ae req-50821ff2-d65b-406b-9697-e65a205918eb tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:05.023444 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-08e4063a-59d1-409a-9507-7b5356a1a2ae req-50821ff2-d65b-406b-9697-e65a205918eb tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2977345 May 25 16:04:05.027466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:04:05.033557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-082eae71-fa24-4483-bdf5-c132a84b3fe2 req-8d4f9d25-81de-4014-b3ca-a0cc844b3ed3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:05.037065 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-082eae71-fa24-4483-bdf5-c132a84b3fe2 req-8d4f9d25-81de-4014-b3ca-a0cc844b3ed3 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1098 time: 0.0901361 May 25 16:04:05.064550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:05.066246 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:04:05.089438 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:04:05.093006 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:05.099101 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-082eae71-fa24-4483-bdf5-c132a84b3fe2 req-d58e2551-0e77-4953-b565-58f90c8aedf8 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0519838 May 25 16:04:05.250678 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-08e4063a-59d1-409a-9507-7b5356a1a2ae req-22b97266-75a9-4201-a6b9-8750d91e1780 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2162120 May 25 16:04:05.279491 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6f553609-9b78-4d33-9455-6b7a0d8ddf46 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "PUT /v2.0/floatingips/36261b1e-35fc-430b-8736-29cd641cfe62 HTTP/1.1" status: 200 len: 992 time: 2.9827471 May 25 16:04:05.279491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:05.368587 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:04:05.415487 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-230112ce-8e99-48c2-a084-5ebe0ef80a53 req-6b8e8cac-910b-4d18-a726-792f53c7aaeb tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:05.417993 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-230112ce-8e99-48c2-a084-5ebe0ef80a53 req-6b8e8cac-910b-4d18-a726-792f53c7aaeb tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.2801964 May 25 16:04:05.510131 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-230112ce-8e99-48c2-a084-5ebe0ef80a53 req-8dc49da8-c4ef-4faf-9df9-ae3b0af49d51 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0858319 May 25 16:04:05.679544 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c4c6a50a-2167-4581-843d-f7f8fb3b18bb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:05.737337 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'status': 'completed', 'code': 200} May 25 16:04:05.851102 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '36261b1e-35fc-430b-8736-29cd641cfe62', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.163', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 115, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:23Z', 'updated_at': '2021-05-25T16:04:03Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:05.855098 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:04:05.866731 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:04:05.867436 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:03:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:05.999056 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-81d96f59-88f5-4bb3-b97e-4bbd0f95630a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.000613 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-81d96f59-88f5-4bb3-b97e-4bbd0f95630a service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1099 time: 0.1547604 May 25 16:04:06.005299 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-0a1f83c3-f60e-4de7-a7d8-fe4f285743fa service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1073024 May 25 16:04:06.124846 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:04:06.255838 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-62601ca2-7f34-45a5-935a-ec63fe1653e7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.257267 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-62601ca2-7f34-45a5-935a-ec63fe1653e7 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.1974332 May 25 16:04:06.368725 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-93cbec6d-cc69-4891-8daf-9711ccd0d4d4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.370827 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-93cbec6d-cc69-4891-8daf-9711ccd0d4d4 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=96b2d1f8431044fca9997bb410be55b8&device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 1098 time: 0.2006350 May 25 16:04:06.391453 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-1fd88422-1c80-451e-9d4f-a95c361c92ba service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1000 time: 0.1242058 May 25 16:04:06.590945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9066357-e271-4766-bb92-dc856470df87 req-76e0e599-e7fa-4202-9abe-fcac4c863d58 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.593559 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d9066357-e271-4766-bb92-dc856470df87 req-76e0e599-e7fa-4202-9abe-fcac4c863d58 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1617265 May 25 16:04:06.640452 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '36261b1e-35fc-430b-8736-29cd641cfe62', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.163', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 115, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:23Z', 'updated_at': '2021-05-25T16:04:03Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:06.649724 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8452c46b-2e32-4bf9-a5ea-f23f163dd138 None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '36261b1e-35fc-430b-8736-29cd641cfe62', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.163', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'fixed_ip_address': '172.20.0.12', 'status': 'DOWN', 'standard_attr_id': 115, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:23Z', 'updated_at': '2021-05-25T16:04:03Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:04:06.658097 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-bc23ecf1-dd1f-4962-b969-4dfed06709dd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.659673 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-bc23ecf1-dd1f-4962-b969-4dfed06709dd service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.2582521 May 25 16:04:06.663989 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d9066357-e271-4766-bb92-dc856470df87 req-842507ee-ebcb-417f-8c58-25590e136e82 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0621920 May 25 16:04:06.826719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-e2c4e7c0-aec0-49b2-a0c0-7221ec89ab08 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.828708 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-e2c4e7c0-aec0-49b2-a0c0-7221ec89ab08 service neutron] 10.222.0.42 "GET /v2.0/networks?id=5e540876-3bc6-4d60-aed5-9bca9dafe63d HTTP/1.1" status: 200 len: 912 time: 0.3206947 May 25 16:04:06.847531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2acdd27b-6414-42cb-bbbd-e109db15a9fa req-a642ac1d-ddbb-404a-ab91-f895b3a24704 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.849981 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2acdd27b-6414-42cb-bbbd-e109db15a9fa req-a642ac1d-ddbb-404a-ab91-f895b3a24704 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1265731 May 25 16:04:06.866983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-44d1e015-bfc9-4e3d-b733-8ed1ce17de13 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:06.868761 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-44d1e015-bfc9-4e3d-b733-8ed1ce17de13 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1248772 May 25 16:04:06.905642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2acdd27b-6414-42cb-bbbd-e109db15a9fa req-e08803c0-f720-43ab-a93b-4f98d2be91f5 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0486515 May 25 16:04:06.991651 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-a9dae0e9-139b-4b7d-b433-fedd7ee7c193 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.2&port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607 HTTP/1.1" status: 200 len: 217 time: 0.0726278 May 25 16:04:06.992339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8c1f76a2-46e6-4899-a716-553539a72956 None None] New status for floating IP 36261b1e-35fc-430b-8736-29cd641cfe62: ACTIVE {{(pid=99546) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:04:07.202716 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-5d0d71cc-4b33-48b3-b871-4182e05e788b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:07.474976 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-5d0d71cc-4b33-48b3-b871-4182e05e788b service neutron] 10.222.0.42 "GET /v2.0/subnets?id=7a4c9393-23ba-4625-b1b4-895efa8276f1 HTTP/1.1" status: 200 len: 851 time: 0.1807325 May 25 16:04:07.474976 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-5dae754b-9bb8-49b1-ad1e-f997f2f11b29 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=5e540876-3bc6-4d60-aed5-9bca9dafe63d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0502851 May 25 16:04:07.474976 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-c20fc6b0-1760-49f5-8789-195752615e09 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4170361 May 25 16:04:07.643172 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-1836efea-afaf-49df-a8af-91790e959635 service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3322170 May 25 16:04:07.684252 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6a8b0c18-d634-4e7b-b342-03782ade55f2 req-73d4abed-af30-4327-bb03-2dc796422059 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2167976 May 25 16:04:07.751748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41684) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:07.804017 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b5f5e2fe-d30d-48f0-bdfd-198e2125fdb3 req-b304bc93-54c9-4f6f-8f1f-417aba4ba1de service neutron] 10.222.0.42 "GET /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1488707 May 25 16:04:07.910051 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f252789-fad1-4bb7-a54b-46e6b196a6fe req-9ae316b0-8d9b-40fd-b760-5100a3638fa4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:07.912577 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f252789-fad1-4bb7-a54b-46e6b196a6fe req-9ae316b0-8d9b-40fd-b760-5100a3638fa4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0944054 May 25 16:04:07.958493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:07.991451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:07.996164 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f252789-fad1-4bb7-a54b-46e6b196a6fe req-8b909c32-3887-43b5-a5f7-d078f95502a8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0766833 May 25 16:04:07.998710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Usage tracker for resource:security_group_rule and tenant:0d2e2c6ae8984f1e922c14b95e74b6e5 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:08.002493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:08.020690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:08.193685 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.275808 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cf5ad9ca-5dc9-4d1a-b1af-07a0e57ce43d req-43754009-ab9c-42ae-8ce9-5bf7077dfe6d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:08.278618 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cf5ad9ca-5dc9-4d1a-b1af-07a0e57ce43d req-43754009-ab9c-42ae-8ce9-5bf7077dfe6d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.2073402 May 25 16:04:08.321197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9650e689-64fb-4259-9c9b-bb3f0294065e req-637144f5-e8ff-4779-b08b-b58630444bbd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:08.323303 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9650e689-64fb-4259-9c9b-bb3f0294065e req-637144f5-e8ff-4779-b08b-b58630444bbd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1870775 May 25 16:04:08.325015 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.332440 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41706) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:08.338335 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'floatingip': {'port_id': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:08.356653 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cf5ad9ca-5dc9-4d1a-b1af-07a0e57ce43d req-b7ce82cc-bc99-4264-933b-51bcae77f6af tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0667300 May 25 16:04:08.416541 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.427821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:08.435312 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.477006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=44c688b3-f3ee-4d5f-ac64-02b4bd961ff4,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:08.489038 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=0d2e2c6ae8984f1e922c14b95e74b6e5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7336223 May 25 16:04:08.489038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.489038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.489038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.489832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.489832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.490197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.490535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.490888 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.491231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.491564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.491668 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41712) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:08.491897 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.492222 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:08.496549 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e279ea12-37b3-455f-8a59-40d1d5d39ad3 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.506693 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:08.668110 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.680390 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.779523 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41730) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:08.800344 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.952420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:08.957278 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-1881db50-7551-4fe9-95b3-e8aee82461b9 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:08.958994 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-1881db50-7551-4fe9-95b3-e8aee82461b9 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.1911108 May 25 16:04:08.977635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-b147fa54-67e1-48f0-9702-0ea8503e47db tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/quotas/ed476c44700b47648958b0d2e020320d HTTP/1.1" status: 200 len: 375 time: 0.0122521 May 25 16:04:09.062780 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.065387 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-530a058a-d9cb-4f5d-8935-e0200f97983f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&fields=id HTTP/1.1" status: 200 len: 255 time: 0.0794334 May 25 16:04:09.164274 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:09.168506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.178216 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Usage tracker for resource:security_group_rule and tenant:b9bb1041cd374283a582970ec1a5bfe1 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:09.191560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:09.194470 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:09.209179 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d12525e0-6faa-49b4-83f7-b8d89fbeeda5 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1357584 May 25 16:04:09.209557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:09.219283 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=44c688b3-f3ee-4d5f-ac64-02b4bd961ff4,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:09.244153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.247663 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/44c688b3-f3ee-4d5f-ac64-02b4bd961ff4 HTTP/1.1" status: 204 len: 168 time: 0.7526448 May 25 16:04:09.247663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.247663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.247663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.247663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.249692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.250018 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.250368 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.250712 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.251169 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.251435 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.251746 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.252079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:04:09.268295 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:09.383146 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-610f7ec1-6a6d-4d4e-b85f-4e5f643c7392 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1659365 May 25 16:04:09.536999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.538831 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.db.resource_extend [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] It took 0.19 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 25 16:04:09.639308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30309cdb-5e45-460a-b9e8-5980ee2ce7f5 req-a090983d-0782-484f-94ef-8fa327186c81 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:09.640818 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-30309cdb-5e45-460a-b9e8-5980ee2ce7f5 req-a090983d-0782-484f-94ef-8fa327186c81 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1131799 May 25 16:04:09.659204 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.668698 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.678462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a605315d-5856-4951-bf17-99b2e2630623 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:09.689848 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-30309cdb-5e45-460a-b9e8-5980ee2ce7f5 req-ac0c9982-3473-46d4-bc64-f93644264aa9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0429986 May 25 16:04:09.732434 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=b9bb1041cd374283a582970ec1a5bfe1&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9495544 May 25 16:04:09.734755 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e71d140b-c62d-4658-ba2c-b4ac0b140d10,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:09.736882 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41748) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:09.738606 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:09.748746 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.913175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0bb55646-5e14-459b-9316-fc3d5635eeb6 req-93af39a0-7c70-442a-b7cb-81e7c653b3bb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 210 time: 0.1078923 May 25 16:04:09.937167 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:09.983553 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:10.036991 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Floating IP 91c9ca1e-ed6d-4008-806e-89380d2d5382 associated. External IP: 172.27.2.179, port: 99600d03-17d9-4c12-98bd-6b71a8bbc8e3. May 25 16:04:10.257972 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:04:10.336692 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:10.341315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b2e861f8-4a59-4fc7-845f-9572b09047a6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:10.355065 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e71d140b-c62d-4658-ba2c-b4ac0b140d10,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:10.358811 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:10.364745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:10.368361 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/e71d140b-c62d-4658-ba2c-b4ac0b140d10 HTTP/1.1" status: 204 len: 168 time: 0.6279385 May 25 16:04:10.416983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:04:10.643242 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:04:10.652523 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:04:10.656437 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:10.661741 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:04:10.662433 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:04:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:10.712661 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-37eec4cc-689a-411e-b19d-df1e6b60d834 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "PUT /v2.0/floatingips/91c9ca1e-ed6d-4008-806e-89380d2d5382 HTTP/1.1" status: 200 len: 991 time: 2.3791018 May 25 16:04:10.713532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:10.782481 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'status': 'completed', 'code': 200} May 25 16:04:10.884477 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41774) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:10.890078 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:04:10.982643 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.089671 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:11.092080 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1e4e38e6-cdfe-4ef1-afb8-e5408990949c req-11d57185-5e54-4302-9ae9-33b054a96309 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:11.094803 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1e4e38e6-cdfe-4ef1-afb8-e5408990949c req-11d57185-5e54-4302-9ae9-33b054a96309 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.2209342 May 25 16:04:11.120668 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Usage tracker for resource:security_group_rule and tenant:cf62b8bcb9624d51abebf697f4d82204 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:11.131101 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.140589 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:11.162950 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:11.207742 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8f041576-ced7-45a4-b40e-d98c77cb10c3 req-3b77b3b8-3cff-43e7-bdd0-180d56a037d4 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 210 time: 0.1105597 May 25 16:04:11.249664 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-c0b1701c-8a24-4265-9e15-75ff8716017b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:11.251254 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-c0b1701c-8a24-4265-9e15-75ff8716017b service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1098 time: 0.3297598 May 25 16:04:11.329840 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1e4e38e6-cdfe-4ef1-afb8-e5408990949c req-5832f5d7-a814-4add-9fb4-12919a199ad4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1206396 May 25 16:04:11.435685 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.458744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-b28552ee-07d2-4203-8ab9-85d64cd984d6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:11.460259 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-b28552ee-07d2-4203-8ab9-85d64cd984d6 service neutron] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 912 time: 0.1978109 May 25 16:04:11.497839 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '91c9ca1e-ed6d-4008-806e-89380d2d5382', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.179', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 138, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:19:6a:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:34Z', 'updated_at': '2021-05-25T16:04:09Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:11.502924 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:04:11.518353 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:04:11.521530 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:04:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:11.689093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.706311 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-71d75738-c606-4c6c-8ece-1f322b605391 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:11.732526 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41788) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:11.743118 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:04:11.744941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=42de45f5-f608-44cc-b6b6-e040d6c7a1b7,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:11.756868 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.772966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:11.785668 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:11.790337 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Usage tracker for resource:security_group_rule and tenant:252c0a11e6114abda3ed39ba914fff83 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:11.794271 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:11.801936 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:11.805974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41792) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:11.806572 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=cf62b8bcb9624d51abebf697f4d82204&name=default HTTP/1.1" status: 200 len: 2979 time: 0.9208968 May 25 16:04:11.811496 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-a4563a44-b4b0-4cbb-a4ff-163cf0d3f5b8 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 999 time: 0.3088548 May 25 16:04:11.852479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41796) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:11.901063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-99462116-f8e9-48a0-b64e-838ff5065abd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:11.902504 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-99462116-f8e9-48a0-b64e-838ff5065abd service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.0788703 May 25 16:04:11.912241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:11.954935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.052248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.077185 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-7f3c35e6-7f7d-4a09-9a48-18f64b3a205b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:12.079074 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-7f3c35e6-7f7d-4a09-9a48-18f64b3a205b service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1517904 May 25 16:04:12.198154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.219885 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.230149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:12.253066 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '91c9ca1e-ed6d-4008-806e-89380d2d5382', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.179', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 138, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:19:6a:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:34Z', 'updated_at': '2021-05-25T16:04:09Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:12.259992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:12.260458 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b08a86a5-be05-48e9-ba6b-86d17f468f00 None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '91c9ca1e-ed6d-4008-806e-89380d2d5382', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.179', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'fixed_ip_address': '172.20.0.9', 'status': 'DOWN', 'standard_attr_id': 138, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:19:6a:7a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:03:34Z', 'updated_at': '2021-05-25T16:04:09Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:04:12.267566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.293424 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Usage tracker for resource:security_group_rule and tenant:4432e1c935ec4533b0e6c1240004e675 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:12.303705 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=252c0a11e6114abda3ed39ba914fff83&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5699089 May 25 16:04:12.307692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41808) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:12.309201 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:12.324229 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1e97b6e5-1ec6-4eda-9c6e-05f2df01bc97,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:12.327665 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:12.328970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:12.333828 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.044s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:12.354641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=42de45f5-f608-44cc-b6b6-e040d6c7a1b7,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:12.358581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:12.369056 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.375821 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/42de45f5-f608-44cc-b6b6-e040d6c7a1b7 HTTP/1.1" status: 204 len: 168 time: 0.5669968 May 25 16:04:12.494332 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-986d1330-4565-431b-9f7b-a10a6513c4d7 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3368273 May 25 16:04:12.511191 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.554782 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-06a77220-c233-44d9-8216-f8483bcc6bc6 req-9ba7f32b-d295-4777-8207-49a739b7b3cb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 210 time: 0.1018145 May 25 16:04:12.682064 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.682197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ff18acbb-f683-489c-bb5d-006cb586a6b0 None None] New status for floating IP 91c9ca1e-ed6d-4008-806e-89380d2d5382: ACTIVE {{(pid=99546) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:04:12.715880 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e8d2c289-ef00-4f50-8192-a9c32c7d8c9f req-f773861a-12a0-4796-87e6-c61b8ba5cb00 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:12.717611 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e8d2c289-ef00-4f50-8192-a9c32c7d8c9f req-f773861a-12a0-4796-87e6-c61b8ba5cb00 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1617723 May 25 16:04:12.747143 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8c40430b-1f2e-4034-9aa4-8827e448a713 req-bf645d92-1dbc-47fb-b7f0-6091da75afff service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2248631 May 25 16:04:12.755141 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.787400 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e8d2c289-ef00-4f50-8192-a9c32c7d8c9f req-d32983c6-84ec-4f1b-af8b-9020fe8f742a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0555041 May 25 16:04:12.839612 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.840202 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.850053 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:12.850160 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155adc83-ae16-47d1-b163-0c5b2ec0f755 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:12.862642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1e97b6e5-1ec6-4eda-9c6e-05f2df01bc97,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:12.866916 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:12.876208 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:12.881562 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/1e97b6e5-1ec6-4eda-9c6e-05f2df01bc97 HTTP/1.1" status: 204 len: 168 time: 0.5649874 May 25 16:04:12.921840 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=4432e1c935ec4533b0e6c1240004e675&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0656686 May 25 16:04:12.921840 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=52a378c4-a9f0-44cb-969c-6aac918259bf,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:12.922308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 41822) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:12.924765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.076s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:12.997318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:13.032582 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:13.250690 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:13.258085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b354858a-3383-44cb-bc37-b3e1e3b94b2d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:13.269042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=52a378c4-a9f0-44cb-969c-6aac918259bf,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:13.271792 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:13.278929 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:13.283643 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/52a378c4-a9f0-44cb-969c-6aac918259bf HTTP/1.1" status: 204 len: 168 time: 0.3601687 May 25 16:04:13.746926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d6cb38ec-3c08-4a03-be64-e01e30a29464 req-f31d1954-7e00-4b3b-b1db-a466e4635836 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 210 time: 0.0471032 May 25 16:04:13.776185 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-6024037e-09f1-4938-b2ac-9423e36a7644 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:13.779485 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-6024037e-09f1-4938-b2ac-9423e36a7644 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.2773612 May 25 16:04:13.824727 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Request body: {'port': {'device_id': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'admin_state_up': True, 'tenant_id': 'ed476c44700b47648958b0d2e020320d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:13.988988 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:13.993392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Usage tracker for resource:port and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:13.997408 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Quota usage for port was recalculated. Used quota:1. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:14.003182 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attempting to reserve 1 items for resource port. Total usage: 1; quota limit: 500; headroom:499 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:14.022354 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-171d1e87-329a-4f74-84bd-6d21ccd2acd7 req-27827a38-de92-4188-aa06-7f3b7c44c539 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:14.031095 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-171d1e87-329a-4f74-84bd-6d21ccd2acd7 req-27827a38-de92-4188-aa06-7f3b7c44c539 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1009998 May 25 16:04:14.077955 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-171d1e87-329a-4f74-84bd-6d21ccd2acd7 req-652f05ed-d786-49b5-bb54-60bbc4859435 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0476058 May 25 16:04:14.136220 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:14.140222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.193633 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41864) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:14.231895 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.250284 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:14.254840 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Usage tracker for resource:security_group_rule and tenant:332fe9c54e7a43e386e7d59d3f7065e5 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:14.258581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:14.276347 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:14.330609 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Allocated IP 172.20.0.3 (9123c0ac-e64c-41a9-8d68-d98246284959/b19f7657-6038-4479-b9ec-9e4e92c69bb0/5c5712c5-651e-4b21-a6fc-b32aff76f3cb) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:04:14.369604 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.445028 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.454568 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a0b88525-d9e5-4f04-80f3-e8fbf1620d91 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:14.502622 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=332fe9c54e7a43e386e7d59d3f7065e5&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3071136 May 25 16:04:14.504817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6bf35f7b-4eb1-47cf-9f56-73ff622a35c9,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:14.507261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 41878) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:14.512165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:14.598101 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.603997 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.636074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.720229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.729069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:14.766051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:14.830915 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Network 9123c0ac-e64c-41a9-8d68-d98246284959 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:04:14.836956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:04:14.841805 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:04:14.850911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:14.851609 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.869296 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:14.899295 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f1ecf0df-19a5-4c1d-bd20-d71308b53271 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:14.961840 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event updated for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:14.967975 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.239s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:15.026557 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6bf35f7b-4eb1-47cf-9f56-73ff622a35c9,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:15.041431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:15.076542 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:15.081619 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/6bf35f7b-4eb1-47cf-9f56-73ff622a35c9 HTTP/1.1" status: 204 len: 168 time: 0.5694761 May 25 16:04:15.125163 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6afc8aac-a45b-4839-8aa7-4cb784c09000 req-90bad83a-9ac8-4fb5-a25e-44fb1d57450b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:15.127605 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6afc8aac-a45b-4839-8aa7-4cb784c09000 req-90bad83a-9ac8-4fb5-a25e-44fb1d57450b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 931 time: 0.1731486 May 25 16:04:15.149077 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3329403 May 25 16:04:15.191672 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6afc8aac-a45b-4839-8aa7-4cb784c09000 req-ded65352-d821-42f2-adb1-32ae310d24b5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0577593 May 25 16:04:15.215696 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Request body: {'port': {'device_id': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:15.393657 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:15.565513 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c85a7c6a-bc93-434b-9481-c36ac820ce13 req-83e9fb7c-c065-4df4-b6e0-85d8c62f2935 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:15.565513 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c85a7c6a-bc93-434b-9481-c36ac820ce13 req-83e9fb7c-c065-4df4-b6e0-85d8c62f2935 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1242986 May 25 16:04:15.643046 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c85a7c6a-bc93-434b-9481-c36ac820ce13 req-e83684b0-5607-4d01-a19b-a68dab9f2e79 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0666332 May 25 16:04:16.025014 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 5c5712c5-651e-4b21-a6fc-b32aff76f3cb {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:16.039547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:16.053127 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Transition to ACTIVE for port object 5c5712c5-651e-4b21-a6fc-b32aff76f3cb will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:04:16.137163 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:16.149055 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a5824f8e-0cc8-4958-9b37-8e367956bfbb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:16.427547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Pushing event updated for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:16.437480 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.285s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:16.449474 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Attempting to bind port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:04:16.449915 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Attempting to bind port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:04:16.450233 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Attempting to bind port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:16.456316 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb on network 9123c0ac-e64c-41a9-8d68-d98246284959 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:04:16.456718 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Attempting to bind port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:16.483803 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 4, 9), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:04:16.484333 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Checking segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:04:16.484965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Bound using segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:04:16.485667 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Bound port: 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:04:16.486242 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:16.537105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-121c5150-45b8-40a2-9d6e-1fe9861ee922 req-c3b50225-2aaf-486c-a918-90d59979269f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:16.541504 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-121c5150-45b8-40a2-9d6e-1fe9861ee922 req-c3b50225-2aaf-486c-a918-90d59979269f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 943 time: 0.1289914 May 25 16:04:16.650202 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-121c5150-45b8-40a2-9d6e-1fe9861ee922 req-d43eb5ff-5d91-4e37-8687-97ce0a19075d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0973456 May 25 16:04:16.685181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:04:16.689072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:04:16.793473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:16.798455 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:16.856364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:16.857122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:16.858520 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] 10.222.0.42 "PUT /v2.0/ports/5c5712c5-651e-4b21-a6fc-b32aff76f3cb HTTP/1.1" status: 200 len: 1093 time: 1.6510792 May 25 16:04:16.862340 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:16.898364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Pushing event updated for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:16.904383 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:16.988563 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2eba2f34-24f9-424a-8638-64191f57e0e6 req-8804a95a-18f0-4a62-a505-2e1799279f6f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:16.999225 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2eba2f34-24f9-424a-8638-64191f57e0e6 req-8804a95a-18f0-4a62-a505-2e1799279f6f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.2173438 May 25 16:04:17.057308 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2eba2f34-24f9-424a-8638-64191f57e0e6 req-604c389a-8410-4fe1-8343-e65d5cc79dc2 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0567982 May 25 16:04:17.155435 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', 'status': 'completed', 'code': 200} May 25 16:04:17.191488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-dacfcde1-7a4c-44c1-8080-332124d8267e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:17.193200 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-dacfcde1-7a4c-44c1-8080-332124d8267e service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 1096 time: 0.1083977 May 25 16:04:17.310091 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-a157e931-6a8c-4bb5-83fc-192b33e5a0fb service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb HTTP/1.1" status: 200 len: 217 time: 0.0982161 May 25 16:04:17.407135 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-bea10d9d-b86d-4fc2-9fab-3848d0f708d9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:17.408750 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-bea10d9d-b86d-4fc2-9fab-3848d0f708d9 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.0790937 May 25 16:04:17.474225 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-34767c07-4d4c-4e21-b30a-7b7e130c93e5 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0553873 May 25 16:04:17.677697 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-4203d047-c3e6-45e8-99f5-a0bd74252af8 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1945951 May 25 16:04:17.890238 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-94ead0b1-539f-4d58-904b-0745db4eaf11 req-f8cecfb9-a917-4030-96aa-a3b9a1fd41de service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1925097 May 25 16:04:18.043506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0524705e-7b75-459b-b7e0-33ef205dec29 req-5720cbcd-d46f-4ed9-aeec-3e7e58d47678 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.045273 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0524705e-7b75-459b-b7e0-33ef205dec29 req-5720cbcd-d46f-4ed9-aeec-3e7e58d47678 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 943 time: 0.1877654 May 25 16:04:18.061805 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8290127-fd39-4030-84f1-ae01216558dc req-5f0e6110-3c7a-4881-a729-a26d825a509a tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.065905 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c8290127-fd39-4030-84f1-ae01216558dc req-5f0e6110-3c7a-4881-a729-a26d825a509a tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.1562626 May 25 16:04:18.147476 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0524705e-7b75-459b-b7e0-33ef205dec29 req-0df55ef5-0c9a-44b7-925c-9253b38f9869 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0949667 May 25 16:04:18.153596 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c8290127-fd39-4030-84f1-ae01216558dc req-b015c63e-10c0-4fa1-9bc8-7ddad17c5f6a tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0751386 May 25 16:04:18.222791 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-e75db772-6621-4392-af79-cbbe22c979b3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.222791 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-e75db772-6621-4392-af79-cbbe22c979b3 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 1096 time: 0.1149249 May 25 16:04:18.358941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbf5e6cb-cd2b-4957-a123-8bead107e651 req-935a06ac-2005-49e7-87f2-0e92659176a2 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.360420 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbf5e6cb-cd2b-4957-a123-8bead107e651 req-935a06ac-2005-49e7-87f2-0e92659176a2 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0881021 May 25 16:04:18.376174 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-bfcad660-d4db-4f46-bd6d-3df9e51c1f12 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.379886 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-bfcad660-d4db-4f46-bd6d-3df9e51c1f12 service neutron] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 915 time: 0.1450114 May 25 16:04:18.406991 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbf5e6cb-cd2b-4957-a123-8bead107e651 req-ff5eeecc-de99-4099-9f4d-2376b67ac177 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404661 May 25 16:04:18.456669 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-6a258e2a-7735-4ca7-8e3c-30598af26d60 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb HTTP/1.1" status: 200 len: 217 time: 0.0626841 May 25 16:04:18.527689 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-6e306cb3-32e8-40be-944c-26b7bc901c0c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:18.529272 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-6e306cb3-32e8-40be-944c-26b7bc901c0c service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.0655425 May 25 16:04:18.586928 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-3be28818-6dff-49f6-a106-faaede2405a1 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0425403 May 25 16:04:18.780693 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42060) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:18.844464 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-f5d878e0-5e47-4df2-a6b3-285405827531 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2397799 May 25 16:04:18.921377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:18.936251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:18.940103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Usage tracker for resource:security_group_rule and tenant:3e1d23ed158d49f5b080576230ec50b7 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:18.942935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:18.949263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:19.005521 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e35580dc-8ab0-40e8-9b8c-fafa588be2b8 req-7ea3c305-b3ba-4700-a75a-0dcb1f42ed56 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1404917 May 25 16:04:19.044935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.130266 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.140347 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6d2196ff-5134-447c-b9e9-76a7f07ca169 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:19.184310 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bc5bc554-87cd-4f29-b0bc-60438695111f,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:19.190710 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=3e1d23ed158d49f5b080576230ec50b7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4074085 May 25 16:04:19.190710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:19.193090 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42078) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:19.271772 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.303083 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.421634 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.426994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:19.442366 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bc5bc554-87cd-4f29-b0bc-60438695111f,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:19.448157 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:19.452676 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:19.453801 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/bc5bc554-87cd-4f29-b0bc-60438695111f HTTP/1.1" status: 204 len: 168 time: 0.2591660 May 25 16:04:19.508843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3949fe6-6bb4-4827-bb31-651c9b0d7cee req-fd036e24-34d2-4a34-a63c-969ccee7ca0c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:19.510405 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3949fe6-6bb4-4827-bb31-651c9b0d7cee req-fd036e24-34d2-4a34-a63c-969ccee7ca0c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 943 time: 0.2286332 May 25 16:04:19.519556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d393d87-af6a-40a0-b945-eddfa334704f req-d0a9f998-fc92-4e83-9353-aa4eb7a35bf5 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:19.523409 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7d393d87-af6a-40a0-b945-eddfa334704f req-d0a9f998-fc92-4e83-9353-aa4eb7a35bf5 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.1294386 May 25 16:04:19.594159 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3949fe6-6bb4-4827-bb31-651c9b0d7cee req-b021b69c-3201-468b-976c-955e2d197e44 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0695317 May 25 16:04:19.599304 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7d393d87-af6a-40a0-b945-eddfa334704f req-d08cb5b5-706d-4ff8-8f6c-8667123c3b11 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0720530 May 25 16:04:19.831548 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-522db00f-d9d5-4284-b182-e6ddebd56742 req-af79febc-64a1-4f4e-aee8-9a69df758651 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:19.831548 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-522db00f-d9d5-4284-b182-e6ddebd56742 req-af79febc-64a1-4f4e-aee8-9a69df758651 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1034050 May 25 16:04:19.908188 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-522db00f-d9d5-4284-b182-e6ddebd56742 req-83ef239f-ca06-4f25-b8ae-8016096cfee5 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0657659 May 25 16:04:20.375360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42120) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:20.392473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.407838 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:20.411692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Usage tracker for resource:security_group_rule and tenant:11c94f3d760b4d4683d6c8b02ada2371 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:20.414520 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:20.420833 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:20.456695 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5c5712c5-651e-4b21-a6fc-b32aff76f3cb details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:04:20.513399 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.564586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.576129 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d1cc57a4-be91-4244-9313-a656581f77bd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:20.617778 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=11c94f3d760b4d4683d6c8b02ada2371&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2427468 May 25 16:04:20.624017 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e73c62f0-ebc1-43c8-9142-7f3b71b96412,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:20.624269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:20.624956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42126) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:20.634206 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:20.652482 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'port_id': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', 'mac_address': 'fa:16:3e:3a:68:d1', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 948, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b19f7657-6038-4479-b9ec-9e4e92c69bb0', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:04:20.704586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.711597 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.732306 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5c5712c5-651e-4b21-a6fc-b32aff76f3cb {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:20.750572 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.755378 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:20.827323 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.829888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:20.916111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cc945451-a639-40ff-9b6f-48c1c954b887 req-dab72272-9cc6-4148-bb65-accc63655aed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:20.917685 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cc945451-a639-40ff-9b6f-48c1c954b887 req-dab72272-9cc6-4148-bb65-accc63655aed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 944 time: 0.1217153 May 25 16:04:20.921825 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:20.925538 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6c265982-474c-4e55-a07c-10c70e70fa88 req-a6eeb067-f784-442f-8b0e-46811d0392c2 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:20.926333 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:20.927079 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6c265982-474c-4e55-a07c-10c70e70fa88 req-a6eeb067-f784-442f-8b0e-46811d0392c2 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.1744840 May 25 16:04:20.930067 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.941276 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-06ed517c-10ee-4183-ad81-268bdfcc0759 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:20.958338 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e73c62f0-ebc1-43c8-9142-7f3b71b96412,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:20.962708 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:20.965547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:20.970638 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/e73c62f0-ebc1-43c8-9142-7f3b71b96412 HTTP/1.1" status: 204 len: 168 time: 0.3434634 May 25 16:04:21.012401 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cc945451-a639-40ff-9b6f-48c1c954b887 req-2af34708-d610-4332-9c94-feaa8ded717d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0843370 May 25 16:04:21.015233 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6c265982-474c-4e55-a07c-10c70e70fa88 req-ac3f4e89-d57d-4067-a7c0-c716a42738f7 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0711329 May 25 16:04:21.238716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-758cebe2-f24b-4820-abcf-d712706d6d59 req-df0594dc-79dc-43e2-8b3a-9312936ff2fa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:21.240314 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-758cebe2-f24b-4820-abcf-d712706d6d59 req-df0594dc-79dc-43e2-8b3a-9312936ff2fa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0886738 May 25 16:04:21.294688 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-758cebe2-f24b-4820-abcf-d712706d6d59 req-e5c76da8-1527-4222-9308-c271e1da8baa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0472364 May 25 16:04:21.375133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-234707c2-b444-4358-b080-3ddd9adf07e9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:21.375846 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-234707c2-b444-4358-b080-3ddd9adf07e9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Synchronizing usage tracker for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:04:21.389720 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['5c5712c5-651e-4b21-a6fc-b32aff76f3cb'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:21.503923 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-234707c2-b444-4358-b080-3ddd9adf07e9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Unset dirty status for tenant:96b2d1f8431044fca9997bb410be55b8 on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:04:21.503923 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-234707c2-b444-4358-b080-3ddd9adf07e9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.42 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 945 time: 0.2242386 May 25 16:04:21.698210 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Deleting port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:04:21.698758 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:21.813865 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42160) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:21.847996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] For port 8c1bac9a-e2d6-43eb-9551-100ddcab0607, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=8c1bac9a-e2d6-43eb-9551-100ddcab0607,segment=NetworkSegment(fcbe700b-26f1-40db-9201-5ec1c3805964),segment_id=fcbe700b-26f1-40db-9201-5ec1c3805964)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:21.848361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:21.958786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:21.979113 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Calling delete_port for 8c1bac9a-e2d6-43eb-9551-100ddcab0607 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:04:21.979547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:21.983803 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Usage tracker for resource:security_group_rule and tenant:acc52caacb914fba89769f5b39cae4a8 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:21.986729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:21.993175 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:22.088972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.261596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.276118 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f802c3ce-1a89-411e-ac53-98992eb81604 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:22.337002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6c1b1d68-2996-4518-a86d-d07ddc378976,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:22.341425 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=acc52caacb914fba89769f5b39cae4a8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5251482 May 25 16:04:22.342184 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.342585 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:22.345757 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42186) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:22.356596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:22.377238 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2a3acd5-d490-4ef2-b113-53933a4a41d5 req-89900d8a-0c75-46eb-bec1-8c236a2d5b92 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:22.378790 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a2a3acd5-d490-4ef2-b113-53933a4a41d5 req-89900d8a-0c75-46eb-bec1-8c236a2d5b92 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 944 time: 0.2200763 May 25 16:04:22.392090 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b9354e3-2051-4fd0-82fa-0006c5f885f6 req-14df11ee-576a-4673-a583-67bc338cc1c9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:22.398067 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6b9354e3-2051-4fd0-82fa-0006c5f885f6 req-14df11ee-576a-4673-a583-67bc338cc1c9 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 867 time: 0.1861668 May 25 16:04:22.422891 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Pushing event deleted for resources: {'Port': ['ID=8c1bac9a-e2d6-43eb-9551-100ddcab0607,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:22.428859 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:22.484089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a2a3acd5-d490-4ef2-b113-53933a4a41d5 req-c12e69f4-f2c1-4595-af45-3f3ca4531f80 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0844836 May 25 16:04:22.487158 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6b9354e3-2051-4fd0-82fa-0006c5f885f6 req-94df6ba9-34ae-42ff-9e81-50c7a7123a41 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/security-groups?id=367321a9-0437-4990-9bd8-2e314882d97c&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0863180 May 25 16:04:22.530492 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['8c1bac9a-e2d6-43eb-9551-100ddcab0607'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:22.554038 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] No DVR routers for this DVR port 8c1bac9a-e2d6-43eb-9551-100ddcab0607 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:04:22.560688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.567991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'name': 'network-vif-deleted', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:22.577157 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.625949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.652388 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f72e6ecf-d665-4fef-b816-75c5ff187313 req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.42 "DELETE /v2.0/ports/8c1bac9a-e2d6-43eb-9551-100ddcab0607 HTTP/1.1" status: 204 len: 173 time: 1.1393270 May 25 16:04:22.749060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f48ccd58-7c7d-43aa-9967-1d212e1851b9 req-f9f53d3a-5499-4bb4-88d5-6fcafc4f23ff tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:22.749060 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f48ccd58-7c7d-43aa-9967-1d212e1851b9 req-f9f53d3a-5499-4bb4-88d5-6fcafc4f23ff tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0935724 May 25 16:04:22.770138 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.775211 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:22.799054 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:22.804799 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/6c1b1d68-2996-4518-a86d-d07ddc378976 HTTP/1.1" status: 204 len: 168 time: 0.4513659 May 25 16:04:22.804799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6c1b1d68-2996-4518-a86d-d07ddc378976,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:22.805130 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f48ccd58-7c7d-43aa-9967-1d212e1851b9 req-2785b0ec-58b4-4359-973e-4b5742c59290 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0451336 May 25 16:04:22.834507 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:22.889089 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '904361ad-111e-47c6-9c15-338ac12347de', 'name': 'network-vif-deleted', 'tag': '8c1bac9a-e2d6-43eb-9551-100ddcab0607', 'status': 'completed', 'code': 200} May 25 16:04:23.722719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42214) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:23.771569 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc3d47fb-97ad-401d-9dfa-321e6b916f37 req-adf163d0-001d-4e58-9cc0-8f4bb5ca14ff tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.1419187 May 25 16:04:23.824549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45913c19-0823-4d41-8ce6-e2740cb24aae req-33b260ec-75df-4d62-97e1-733a09678452 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:23.832919 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-45913c19-0823-4d41-8ce6-e2740cb24aae req-33b260ec-75df-4d62-97e1-733a09678452 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 944 time: 0.1641569 May 25 16:04:23.834554 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:23.865819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:23.871759 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Usage tracker for resource:security_group_rule and tenant:6a8357113fce4815a4bc18a25d82e186 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:23.876642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:23.883793 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:23.934891 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-45913c19-0823-4d41-8ce6-e2740cb24aae req-4190388b-0075-47e7-85f3-bc5a0474b027 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0978940 May 25 16:04:23.991216 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.117888 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.118252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7068f0e-34bb-4621-baee-b6acb6160edf req-264bbe19-f32c-4b87-943f-5b716f95eaca tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:24.119808 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7068f0e-34bb-4621-baee-b6acb6160edf req-264bbe19-f32c-4b87-943f-5b716f95eaca tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0867574 May 25 16:04:24.127864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:24.164781 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7068f0e-34bb-4621-baee-b6acb6160edf req-0165af79-1683-4d4e-bc58-f4da44486bf0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0390134 May 25 16:04:24.168193 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=6a8357113fce4815a4bc18a25d82e186&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4418755 May 25 16:04:24.170347 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4ba6c5d2-6192-4392-86d1-9eb03db877c3,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:24.171268 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42224) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:24.173565 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:24.237165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.263824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.392752 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.396562 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d85a979f-44e0-4e05-9227-077ebbbb5b3b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:24.411627 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4ba6c5d2-6192-4392-86d1-9eb03db877c3,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:24.413991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:24.425164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.426499 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/4ba6c5d2-6192-4392-86d1-9eb03db877c3 HTTP/1.1" status: 204 len: 168 time: 0.2541230 May 25 16:04:24.449211 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5c5712c5-651e-4b21-a6fc-b32aff76f3cb up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:04:24.576064 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:04:24.581747 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:04:24.582193 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.707486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.737773 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:24.748922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:24.792675 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', 'code': 200} May 25 16:04:24.827522 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:24.828794 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 8c1bac9a-e2d6-43eb-9551-100ddcab0607 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:04:24.866060 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:24.903719 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 8c1bac9a-e2d6-43eb-9551-100ddcab0607 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:04:24.906577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 8c1bac9a-e2d6-43eb-9551-100ddcab0607 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:04:24.985162 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:25.004052 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:25.185662 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-48e6bfe7-5800-43da-9f9e-07c59a8fe781 req-cf0ea22f-2430-49bb-9f8d-906b00291170 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] 10.222.0.82 "GET /v2.0/ports?device_id=904361ad-111e-47c6-9c15-338ac12347de HTTP/1.1" status: 200 len: 210 time: 0.0989969 May 25 16:04:25.294847 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b629658-76f2-4382-8d24-5860317c133f req-344e6c1f-b173-4375-94ca-b2d7867b5d00 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:25.296392 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b629658-76f2-4382-8d24-5860317c133f req-344e6c1f-b173-4375-94ca-b2d7867b5d00 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.1297193 May 25 16:04:25.347439 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b629658-76f2-4382-8d24-5860317c133f req-41870f09-1bd9-45fd-87c3-12c02e4cd480 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0442791 May 25 16:04:25.466254 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d6ffbf9-077a-4bf9-9c91-80329a0de087 req-8a75736c-0d72-4565-b85f-10b6c699d350 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:25.467925 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2d6ffbf9-077a-4bf9-9c91-80329a0de087 req-8a75736c-0d72-4565-b85f-10b6c699d350 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 1098 time: 0.0846097 May 25 16:04:25.537094 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2d6ffbf9-077a-4bf9-9c91-80329a0de087 req-5d0fdf5b-3f38-41b9-951d-7907bb7fa343 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0531719 May 25 16:04:25.582932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fe2d9c9a-f3e9-4b24-905f-37a1cfc87014 req-7a1a42cf-cf5a-4f4b-a39d-ce4cd98d2a72 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:25.584550 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fe2d9c9a-f3e9-4b24-905f-37a1cfc87014 req-7a1a42cf-cf5a-4f4b-a39d-ce4cd98d2a72 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1287928 May 25 16:04:25.631299 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fe2d9c9a-f3e9-4b24-905f-37a1cfc87014 req-21e9bbfe-529e-488a-95f4-82c7a636ef85 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0401542 May 25 16:04:26.620538 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c086037b-f09f-447b-8ec7-70d247b7661d req-ed4e6d9b-3c23-48cf-95ae-e121c56b9713 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:26.622133 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c086037b-f09f-447b-8ec7-70d247b7661d req-ed4e6d9b-3c23-48cf-95ae-e121c56b9713 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.1234112 May 25 16:04:26.805418 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c086037b-f09f-447b-8ec7-70d247b7661d req-e049e7ab-154e-4082-aa05-0ad814698a32 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1751842 May 25 16:04:26.939756 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6b293843-185e-46ee-8244-76b2a77e9927 req-5dc522ff-b843-43d1-8dc2-0bc6826e0f13 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:26.941285 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6b293843-185e-46ee-8244-76b2a77e9927 req-5dc522ff-b843-43d1-8dc2-0bc6826e0f13 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1294820 May 25 16:04:26.994208 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6b293843-185e-46ee-8244-76b2a77e9927 req-c25acab3-f801-40dc-bf41-a6b7e1f698b1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0471478 May 25 16:04:27.996697 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2df3adcd-edbe-4eb3-8464-3c8e7c3e70c8 req-b6748538-036e-4ae5-bd4c-ee18b33231f8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:27.998301 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2df3adcd-edbe-4eb3-8464-3c8e7c3e70c8 req-b6748538-036e-4ae5-bd4c-ee18b33231f8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.0862005 May 25 16:04:28.049171 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2df3adcd-edbe-4eb3-8464-3c8e7c3e70c8 req-e6da1276-7a0e-406b-985d-0f11d2f7c53d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0447078 May 25 16:04:28.207304 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-004b444b-3320-4c51-aeed-bae461604d0e req-8d8f42da-cb13-4202-8224-cf790f414732 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:28.208782 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-004b444b-3320-4c51-aeed-bae461604d0e req-8d8f42da-cb13-4202-8224-cf790f414732 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0926328 May 25 16:04:28.256617 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-004b444b-3320-4c51-aeed-bae461604d0e req-9c265575-8dea-4d21-8c47-41334f506436 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0417154 May 25 16:04:28.927275 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42372) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:28.999002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Deleting subnet 7a4c9393-23ba-4625-b1b4-895efa8276f1 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:04:29.003164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.058207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.065405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.382266 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42382) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:29.452299 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.461535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ad3867d8-efae-44f0-a4ce-a55f5878d0dc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:29.504704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=7a4c9393-23ba-4625-b1b4-895efa8276f1,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:29.511507 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.514154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:29.524452 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/7a4c9393-23ba-4625-b1b4-895efa8276f1 HTTP/1.1" status: 204 len: 168 time: 0.5934930 May 25 16:04:29.525427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42394) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:29.536881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b451528-55df-4d26-8050-bdcdcf6a4337 req-6e32554d-7024-48ad-bc0e-f5eb949cd499 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:29.543607 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b451528-55df-4d26-8050-bdcdcf6a4337 req-6e32554d-7024-48ad-bc0e-f5eb949cd499 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1604707 May 25 16:04:29.671743 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.688060 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:29.692261 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Usage tracker for resource:security_group_rule and tenant:2954597aae6f4105bce55d02879ac25c is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:29.695986 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:29.701512 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.705799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:29.714123 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b451528-55df-4d26-8050-bdcdcf6a4337 req-7ea838e0-42b5-4526-8191-b517675ed8f0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0681324 May 25 16:04:29.820714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.849147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.866392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > segment before_delete 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.866717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.867512 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.867965 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > port after_create 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.868282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > port after_delete 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.868632 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > port after_update 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.868949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > segment after_delete 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.869263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.869575 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.869891 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet after_update 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:29.907198 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:29.915921 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-96d15e2c-78d8-4c26-9800-05f255fb9c36 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:29.956671 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=2954597aae6f4105bce55d02879ac25c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5733120 May 25 16:04:29.958372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9e3f5417-d035-4162-a376-f44afa7e66ea,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:29.962914 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:29.963497 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42400) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:30.033848 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.043687 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:30.044543 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:30.044543 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet before_create 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:30.044814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Subscribe: > subnet before_update 55550000 {{(pid=99545) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:04:30.045071 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['5e540876-3bc6-4d60-aed5-9bca9dafe63d']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:30.059703 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'fcbe700b-26f1-40db-9201-5ec1c3805964') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:30.060026 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'fcbe700b-26f1-40db-9201-5ec1c3805964') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:04:30.068537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.079220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.099498 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.129181 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Releasing vlan 901 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:04:30.131321 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: from cryptography.utils import int_from_bytes May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: from cryptography.utils import int_from_bytes May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:04:30.154113 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:04:30.232394 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.236407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-53e972a4-f2e3-42e1-8bca-2f4a714d70fe admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:30.250165 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9e3f5417-d035-4162-a376-f44afa7e66ea,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:30.254416 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:30.260837 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.262090 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/9e3f5417-d035-4162-a376-f44afa7e66ea HTTP/1.1" status: 204 len: 168 time: 0.2973502 May 25 16:04:30.370044 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.372291 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:30.379011 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:30.402820 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Pushing event deleted for resources: {'Network': ['ID=5e540876-3bc6-4d60-aed5-9bca9dafe63d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:30.406719 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:30.506802 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-fb88658f-ea55-4e84-8560-39d07f4a3afe admin admin] 10.222.0.82 "DELETE /v2.0/networks/5e540876-3bc6-4d60-aed5-9bca9dafe63d HTTP/1.1" status: 204 len: 168 time: 0.9796412 May 25 16:04:30.928049 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20c8bf3c-e28d-4488-a835-559ed8a0cef8 req-ddd5d00b-cf5c-4f79-b5fe-fb304250b557 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:30.930018 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20c8bf3c-e28d-4488-a835-559ed8a0cef8 req-ddd5d00b-cf5c-4f79-b5fe-fb304250b557 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0928466 May 25 16:04:30.951240 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42434) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:30.980555 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20c8bf3c-e28d-4488-a835-559ed8a0cef8 req-fe52109e-53dd-425e-b9c1-797e4fc1ef64 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0450513 May 25 16:04:31.022177 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-78c2fa1c-1a79-4cce-8336-a1edc23f6528 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=96b2d1f8431044fca9997bb410be55b8&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0689015 May 25 16:04:31.025414 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42438) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:31.089093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.115932 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.249569 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.254358 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:31.272119 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=367321a9-0437-4990-9bd8-2e314882d97c,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:31.275393 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:31.456860 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42454) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:31.500062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.501177 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/367321a9-0437-4990-9bd8-2e314882d97c HTTP/1.1" status: 204 len: 168 time: 0.4748724 May 25 16:04:31.595715 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1314659583-network', 'tenant_id': '9e6e907c5841468a85bc59f21b26372d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:31.600323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:31.604794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Usage tracker for resource:network and tenant:9e6e907c5841468a85bc59f21b26372d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:31.609758 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:31.615843 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:31.718611 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Made reservation on behalf of 9e6e907c5841468a85bc59f21b26372d for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:31.723034 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.737418 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.754097 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:31.758084 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Usage tracker for resource:security_group_rule and tenant:9e6e907c5841468a85bc59f21b26372d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:31.761640 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:31.767842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:31.882026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.947265 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:31.952581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a55163a5-d34c-45b1-ba35-7503b31bd8b2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:31.955708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-1281a51a-1387-42fd-ab3f-6d15d744f0db tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:31.957322 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-1281a51a-1387-42fd-ab3f-6d15d744f0db tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.1685522 May 25 16:04:31.979591 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-036d89ac-ea94-4d54-a6e4-624615ca6bdf tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/quotas/ed476c44700b47648958b0d2e020320d HTTP/1.1" status: 200 len: 375 time: 0.0144734 May 25 16:04:31.981081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] vlan segment allocate from pool success with {'vlan_id': 918, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:04:31.996067 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7547d244-8c01-4ddb-8286-ed592f9280b0,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:32.002022 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:32.003391 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:32.003995 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Added segment d91de080-4e40-4f71-aea3-f86a8f72864b of type vlan for network 54cf369d-e27b-4d90-a4f4-d729a1eb711e May 25 16:04:32.041878 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:32.093944 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-3e6482a0-89a8-4fe8-922d-d84cd89828a9 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1105776 May 25 16:04:32.192849 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:32.194319 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:32.195047 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:32.201231 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-83d5974f-6d8e-4908-b6ea-aabf273238ab admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:32.305818 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Pushing event updated for resources: {'Network': ['ID=54cf369d-e27b-4d90-a4f4-d729a1eb711e,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:32.318367 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:32.321945 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:04:32.348340 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-5ff58c68-da87-4d26-84e4-4646631fe9d6 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2467999 May 25 16:04:32.478904 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 1.0207243 May 25 16:04:32.483967 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42496) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:32.491339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Request body: {'subnet': {'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1314659583-subnet', 'tenant_id': '9e6e907c5841468a85bc59f21b26372d', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:32.496917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:32.500638 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-267831d9-ae65-483e-b024-f039e530b00f service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1448705 May 25 16:04:32.504334 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Usage tracker for resource:subnet and tenant:9e6e907c5841468a85bc59f21b26372d is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:32.507578 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:32.513713 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:32.632661 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Made reservation on behalf of 9e6e907c5841468a85bc59f21b26372d for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:32.635774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:33.207544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:33.214996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-36f75d49-64bd-45a1-bdc3-80443e9c5553 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:33.285972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Pushing event updated for resources: {'Subnet': ['ID=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:33.290313 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:33.297492 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Checking segment: {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:04:33.297840 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Network hostable DHCP agents. Network: 54cf369d-e27b-4d90-a4f4-d729a1eb711e, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:04:33.386428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:04:33.484068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 54cf369d-e27b-4d90-a4f4-d729a1eb711e) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:04:33.503787 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:33.503989 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-0c0c2f07-89c7-4290-8bcf-bd5a4d3293e4 None None] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:04:33.504531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:33.596676 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.1116478 May 25 16:04:33.730367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-cc1fadfc-9cbb-48da-a7df-9d45ea14776d None None] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:04:34.332711 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74120458-3590-4316-bdd0-b8e8c33b4a17 req-532fe342-308b-40d0-8a0b-f87cbcaa7219 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.0591259 May 25 16:04:34.377735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eda40095-7d8e-40bc-83ba-8c588af0df1b req-8326c904-6d41-41d1-a5a7-7f328dcfdb7a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:34.379670 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eda40095-7d8e-40bc-83ba-8c588af0df1b req-8326c904-6d41-41d1-a5a7-7f328dcfdb7a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1423395 May 25 16:04:34.435031 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eda40095-7d8e-40bc-83ba-8c588af0df1b req-940e0d54-5beb-48f4-a911-0017d67a5cd4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0493836 May 25 16:04:35.074424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-81461fc4-4985-455e-8ead-4dae026714c1 req-d3507f40-a077-4d14-9d0a-351d205bbb8e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:35.075973 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-81461fc4-4985-455e-8ead-4dae026714c1 req-d3507f40-a077-4d14-9d0a-351d205bbb8e service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 1098 time: 0.0764728 May 25 16:04:35.124406 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-81461fc4-4985-455e-8ead-4dae026714c1 req-c310c930-38c1-4f7b-a9a9-561418c9c246 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400951 May 25 16:04:35.666789 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-bb47966d-6f8f-4dd5-9217-da53e4a3378f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:35.770205 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-749898e6-33c6-4d7b-9281-2d13779950fc req-4201a073-3653-435c-8177-29be0f20c874 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.0473418 May 25 16:04:35.968979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e7d5642-52a4-40f1-ad47-f2680516c9e0 req-ca2312a2-5836-4892-88f3-1a880ed1323b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:35.970511 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e7d5642-52a4-40f1-ad47-f2680516c9e0 req-ca2312a2-5836-4892-88f3-1a880ed1323b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.0886240 May 25 16:04:36.029275 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e7d5642-52a4-40f1-ad47-f2680516c9e0 req-fe4c9db8-2871-493d-9498-447a63b73c86 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0522263 May 25 16:04:36.200855 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-81263c3c-8fec-4787-9859-b8e93daca80a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:36.202355 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-81263c3c-8fec-4787-9859-b8e93daca80a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 809 time: 0.1699841 May 25 16:04:36.217404 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-9225b760-f5a3-4b18-8fb2-d91c5453a1cf tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:36.218140 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-3a1ad44e-9cf7-4c1e-93bd-baf42b6a5517 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/quotas/9e6e907c5841468a85bc59f21b26372d HTTP/1.1" status: 200 len: 375 time: 0.0100038 May 25 16:04:36.218854 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-9225b760-f5a3-4b18-8fb2-d91c5453a1cf tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.1503408 May 25 16:04:36.244576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Request body: {'port': {'device_id': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'admin_state_up': True, 'tenant_id': 'ed476c44700b47648958b0d2e020320d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:36.272107 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-603d5ea0-78f3-4da6-a3da-d28632ef7fa1 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0478079 May 25 16:04:36.383941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:36.387808 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Usage tracker for resource:port and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:36.391132 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:36.396940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:36.421408 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-63f57914-3704-4b32-b85c-8300d9b5f8cc service neutron] 10.222.0.82 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1409962 May 25 16:04:36.529225 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:36.532646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:36.563186 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-1277c128-6784-48df-8d4c-f8a08761dd04 service neutron] 10.222.0.82 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1353679 May 25 16:04:36.764908 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Allocated IP 172.20.0.8 (9123c0ac-e64c-41a9-8d68-d98246284959/b19f7657-6038-4479-b9ec-9e4e92c69bb0/2206a32c-5500-4208-9123-7aab17fc53c7) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:04:36.934660 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e88a9ef2-6fa2-405f-aa70-f87da1798d1b req-439e362d-e2d1-466d-8499-282605de98b9 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0468028 May 25 16:04:37.043082 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:37.140084 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7d87a636-6f51-4aa1-a923-2a6e5abbbe10 req-cc7f8db1-7764-4a45-9c5d-dfa49ce10c84 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.0442529 May 25 16:04:37.147396 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:37.155730 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-22b8aa27-ce8f-4990-bf01-e132bad6208b tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:37.238080 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Network 9123c0ac-e64c-41a9-8d68-d98246284959 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:04:37.241396 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:04:37.245966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:04:37.251571 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:37.252247 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:37.273894 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event updated for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:37.290224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:37.334133 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f931667c-fd60-4868-89a4-d0a78ae553db req-1b255817-16c1-45c7-a3fe-779ee1741098 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:37.335556 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f931667c-fd60-4868-89a4-d0a78ae553db req-1b255817-16c1-45c7-a3fe-779ee1741098 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1757114 May 25 16:04:37.372816 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1322539 May 25 16:04:37.397352 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f931667c-fd60-4868-89a4-d0a78ae553db req-0d77fa3d-5493-4022-9c95-23d86df24e53 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0484424 May 25 16:04:37.596417 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f095530f-7f22-4be9-a72f-275420e0238a req-29290f71-c28b-47bd-b4e4-1db50091d78c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:37.598126 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f095530f-7f22-4be9-a72f-275420e0238a req-29290f71-c28b-47bd-b4e4-1db50091d78c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0879471 May 25 16:04:37.649279 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f095530f-7f22-4be9-a72f-275420e0238a req-617251bc-c9e2-48f3-95f7-c2735013f4ec tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0436497 May 25 16:04:38.031197 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ae9fa75-753f-4816-a182-d5f50a638a75 req-967a482c-7033-4f3d-a1a2-857c80c39b87 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0437429 May 25 16:04:38.140073 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:38.461146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-15715f7f-1065-4359-9e68-33b196554431 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:39.073233 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:40.315360 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc', '5c5712c5-651e-4b21-a6fc-b32aff76f3cb'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:40.359133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Request body: {'port': {'device_id': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:40.452859 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:40.493549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:40.710459 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa368e88-9ef2-4a2d-b28f-2d2a03d5f2d4 req-247e12bc-85c7-4a7e-931b-e46e807f5466 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:40.727473 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aa368e88-9ef2-4a2d-b28f-2d2a03d5f2d4 req-247e12bc-85c7-4a7e-931b-e46e807f5466 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 931 time: 0.1663146 May 25 16:04:40.820405 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7c4ea201-db23-49db-806e-9a15928cd5a9 req-b3a9859b-d1f4-42de-8b1d-0506ec61e27e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0557702 May 25 16:04:40.835066 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aa368e88-9ef2-4a2d-b28f-2d2a03d5f2d4 req-71f37ec2-0794-46af-a25b-f8059009d4b8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1066103 May 25 16:04:41.058531 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 2206a32c-5500-4208-9123-7aab17fc53c7 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:41.066190 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.091633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Transition to ACTIVE for port object 2206a32c-5500-4208-9123-7aab17fc53c7 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:04:41.208965 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.209365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.213193 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d36cc63f-af07-4f90-b4c4-1afc3ccff328 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:41.343213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Pushing event updated for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:41.371208 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:41.400438 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Attempting to bind port 2206a32c-5500-4208-9123-7aab17fc53c7 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:04:41.400827 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Attempting to bind port 2206a32c-5500-4208-9123-7aab17fc53c7 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:04:41.401153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Attempting to bind port 2206a32c-5500-4208-9123-7aab17fc53c7 on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:41.405918 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Port 2206a32c-5500-4208-9123-7aab17fc53c7 on network 9123c0ac-e64c-41a9-8d68-d98246284959 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:04:41.406226 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Attempting to bind port 2206a32c-5500-4208-9123-7aab17fc53c7 on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:41.411243 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 4, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:04:41.411572 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Checking segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:04:41.411947 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Bound using segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:04:41.412363 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Bound port: 2206a32c-5500-4208-9123-7aab17fc53c7, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:04:41.412704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.505361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:04:41.508802 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2206a32c-5500-4208-9123-7aab17fc53c7,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:04:41.609754 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.613647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:41.671900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:41.672530 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:41.673919 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] 10.222.0.42 "PUT /v2.0/ports/2206a32c-5500-4208-9123-7aab17fc53c7 HTTP/1.1" status: 200 len: 1093 time: 1.3199024 May 25 16:04:41.674762 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:41.699215 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Pushing event updated for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:41.705512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:41.778234 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7', 'status': 'completed', 'code': 200} May 25 16:04:42.031614 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-3f456319-9964-4349-baa9-89d608d7c905 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.033086 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-3f456319-9964-4349-baa9-89d608d7c905 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 1096 time: 0.1870048 May 25 16:04:42.108912 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4da2da2a-1c4c-467e-a490-9002794e4603 req-350314bb-734b-48b5-a58f-624f46c4d6ff tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0679193 May 25 16:04:42.146481 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9f605dea-04a5-4826-b9c4-58f3ce660ae1 req-f44dfdd9-d73d-4b1c-b688-49763f38c386 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.146481 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9f605dea-04a5-4826-b9c4-58f3ce660ae1 req-f44dfdd9-d73d-4b1c-b688-49763f38c386 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.1431561 May 25 16:04:42.204932 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9f605dea-04a5-4826-b9c4-58f3ce660ae1 req-8c1c0cfa-f575-4636-b41a-338e5f718886 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0554078 May 25 16:04:42.232515 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-aa609e42-4460-48b5-9001-1690e5243192 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.234368 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-aa609e42-4460-48b5-9001-1690e5243192 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=ed476c44700b47648958b0d2e020320d&shared=False HTTP/1.1" status: 200 len: 915 time: 0.1650867 May 25 16:04:42.461005 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-07123caf-7cc7-4d89-b1e6-4ab2452922d9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.462712 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2bd90688-e131-45da-b08e-4c1d16503a26 req-07123caf-7cc7-4d89-b1e6-4ab2452922d9 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2191250 May 25 16:04:42.521580 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-0db08069-cb26-4b22-9c40-6ef23f8ff757 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.522294 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-023a4d47-12fa-438f-abf2-634e379d960c req-0db08069-cb26-4b22-9c40-6ef23f8ff757 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Synchronizing usage tracker for tenant:9e6e907c5841468a85bc59f21b26372d on resource:network {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:04:42.628990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-023a4d47-12fa-438f-abf2-634e379d960c req-0db08069-cb26-4b22-9c40-6ef23f8ff757 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Unset dirty status for tenant:9e6e907c5841468a85bc59f21b26372d on resource:network {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:04:42.630284 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-0db08069-cb26-4b22-9c40-6ef23f8ff757 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 809 time: 0.2761989 May 25 16:04:42.640356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d8834be0-602e-4654-835d-dbcb70875e62 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.642147 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-d8834be0-602e-4654-835d-dbcb70875e62 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 1096 time: 0.0768435 May 25 16:04:42.684006 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Request body: {'port': {'device_id': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e', 'admin_state_up': True, 'tenant_id': '9e6e907c5841468a85bc59f21b26372d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:42.788674 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-37aae06e-d9ff-4a23-ba2d-51fefbff99fa service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=2206a32c-5500-4208-9123-7aab17fc53c7 HTTP/1.1" status: 200 len: 217 time: 0.1088347 May 25 16:04:42.826783 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a03f83e-eeae-4b59-b285-28c5649fa9f6 req-8da1807a-3f87-4258-a896-f8c491cf0695 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.828318 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0a03f83e-eeae-4b59-b285-28c5649fa9f6 req-8da1807a-3f87-4258-a896-f8c491cf0695 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1019008 May 25 16:04:42.939738 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-c72de689-c1d6-411d-82fb-6718e8486276 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:42.940981 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-c72de689-c1d6-411d-82fb-6718e8486276 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.0998294 May 25 16:04:43.009326 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0a03f83e-eeae-4b59-b285-28c5649fa9f6 req-4d6de056-e204-4b9c-8527-2e33f740f186 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1744466 May 25 16:04:43.053835 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:43.060491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Usage tracker for resource:port and tenant:9e6e907c5841468a85bc59f21b26372d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:43.065515 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:43.071861 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-955ca094-9667-4533-b50b-873be1e81c2b req-e2c3da63-0376-4d35-a5fa-7e00daaccf65 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:43.073342 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-e2c3da63-0376-4d35-a5fa-7e00daaccf65 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.2185159 May 25 16:04:43.082066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:43.086784 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-ddd9fb1c-aed1-48e7-9c71-9971a1a32a77 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1251993 May 25 16:04:43.229206 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Made reservation on behalf of 9e6e907c5841468a85bc59f21b26372d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:43.246692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:43.447699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-347e3804-4654-4d0d-adea-fcd47e112f84 req-96b76163-cf24-4add-818f-83afa003a9e2 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0721071 May 25 16:04:43.486052 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-955ca094-9667-4533-b50b-873be1e81c2b req-0fc49653-f239-42ec-be22-d0fce71f5934 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:43.491428 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-0fc49653-f239-42ec-be22-d0fce71f5934 service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.3862543 May 25 16:04:43.628933 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-47fc6b38-46f3-45ef-b088-f641f898c2af service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5216606 May 25 16:04:43.649479 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-5abf3f09-8f2d-4cb7-81ce-79bfc8b221e9 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.0792339 May 25 16:04:43.733240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c2e5bda3-3331-4d55-9620-01b9b1de1a4a req-c04d9207-6561-45f7-bee9-3c614187f060 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:43.734786 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c2e5bda3-3331-4d55-9620-01b9b1de1a4a req-c04d9207-6561-45f7-bee9-3c614187f060 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.2327638 May 25 16:04:43.738603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-955ca094-9667-4533-b50b-873be1e81c2b req-cd48bd1f-16ff-4ba6-8e73-36d6d2a9f9c8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:43.740025 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-cd48bd1f-16ff-4ba6-8e73-36d6d2a9f9c8 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.0795348 May 25 16:04:43.754554 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Allocated IP 172.20.0.9 (54cf369d-e27b-4d90-a4f4-d729a1eb711e/4fa131e2-5aa0-4f62-bf74-2a67a79ad81c/08ee0981-245d-4233-9301-2d6fa104b520) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:04:43.807671 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-1d0bbd78-b738-492b-a320-fd06fbf48153 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0471566 May 25 16:04:43.835236 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-53f88553-0794-475a-bc14-a7fa84bb973d service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1951270 May 25 16:04:43.863767 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c2e5bda3-3331-4d55-9620-01b9b1de1a4a req-f6d180ca-657a-490b-9b21-b0b3a672bc02 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1191263 May 25 16:04:44.041040 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-e6a6b733-d8fa-4dd9-90c3-bc6337c0e22a service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2192907 May 25 16:04:44.056540 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.057898 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42750) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:44.197737 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.209025 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-15a06aca-5024-411b-a94a-5ffd52be8a5b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:44.263286 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.300710 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:04:44.304683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:04:44.309742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:04:44.315522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:44.316189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.332499 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Pushing event updated for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:44.338255 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:44.388558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:44.396683 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Usage tracker for resource:security_group_rule and tenant:ffe5ca75df3f4f1e829ab860f3cd5a40 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:44.401051 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-955ca094-9667-4533-b50b-873be1e81c2b req-fc8cfea3-7059-4f26-aa7a-c96e16b9e34d service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3334432 May 25 16:04:44.404556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:44.412405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:44.435082 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1515b94-4274-49ed-9ee9-97fc38aade6a req-8a682012-f342-44e6-8827-1c0317729a59 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:44.436689 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1515b94-4274-49ed-9ee9-97fc38aade6a req-8a682012-f342-44e6-8827-1c0317729a59 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2682593 May 25 16:04:44.484900 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1515b94-4274-49ed-9ee9-97fc38aade6a req-74f41a9f-6ef9-4a28-8659-f2fce3354d33 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416536 May 25 16:04:44.485964 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.8066082 May 25 16:04:44.517871 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Request body: {'port': {'device_id': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:44.608087 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.676674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.688390 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:44.701646 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7250b0b0-a38d-4ca9-8005-58093799df80 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:44.745641 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=ffe5ca75df3f4f1e829ab860f3cd5a40&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6852801 May 25 16:04:44.749559 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42764) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:44.757425 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=fe227393-ec13-48a3-83fc-884ccf2e6462,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:44.760952 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:44.780972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68bae16d-a171-49dd-bdf6-34e87cb65f44 req-abb77b5c-f9c4-40ef-b424-a38e4e587030 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:44.784217 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68bae16d-a171-49dd-bdf6-34e87cb65f44 req-abb77b5c-f9c4-40ef-b424-a38e4e587030 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 931 time: 0.1487870 May 25 16:04:44.831463 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68bae16d-a171-49dd-bdf6-34e87cb65f44 req-fe942fe9-a4a1-4f9d-acaf-398d226265bd tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0425019 May 25 16:04:44.954047 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.002796 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.047352 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b3ed6db9-f9d3-40da-96a1-84aed18170ed req-c4d1ba27-7b3b-4dbb-af43-5250119b442b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:45.049075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b3ed6db9-f9d3-40da-96a1-84aed18170ed req-c4d1ba27-7b3b-4dbb-af43-5250119b442b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.0860686 May 25 16:04:45.108606 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b3ed6db9-f9d3-40da-96a1-84aed18170ed req-dfb36701-56dc-4142-b9c9-e6e7d9044194 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0470216 May 25 16:04:45.155599 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 08ee0981-245d-4233-9301-2d6fa104b520 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:45.165836 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.227797 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Transition to ACTIVE for port object 08ee0981-245d-4233-9301-2d6fa104b520 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:04:45.228916 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.233070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4d339870-3e91-4d71-8659-af74e53ece0f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:45.248521 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=fe227393-ec13-48a3-83fc-884ccf2e6462,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:45.252371 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:45.256187 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.257367 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/fe227393-ec13-48a3-83fc-884ccf2e6462 HTTP/1.1" status: 204 len: 168 time: 0.5068698 May 25 16:04:45.300684 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.306569 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:45.492166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Pushing event updated for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:45.495810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:45.509220 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Attempting to bind port 08ee0981-245d-4233-9301-2d6fa104b520 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:04:45.511528 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Attempting to bind port 08ee0981-245d-4233-9301-2d6fa104b520 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:04:45.511528 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Attempting to bind port 08ee0981-245d-4233-9301-2d6fa104b520 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:45.516007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Port 08ee0981-245d-4233-9301-2d6fa104b520 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:04:45.516373 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Attempting to bind port 08ee0981-245d-4233-9301-2d6fa104b520 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:04:45.522795 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 4, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:04:45.523232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Checking segment: {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:04:45.524292 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Bound using segment: {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:04:45.524788 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Bound port: 08ee0981-245d-4233-9301-2d6fa104b520, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:04:45.525207 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.625726 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:04:45.629522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=08ee0981-245d-4233-9301-2d6fa104b520,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:04:45.683750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.689343 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-12241f8f-6224-4b64-954e-e0151141e13a service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:45.752609 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:45.753130 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:45.754918 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] 10.222.0.42 "PUT /v2.0/ports/08ee0981-245d-4233-9301-2d6fa104b520 HTTP/1.1" status: 200 len: 1093 time: 1.2450488 May 25 16:04:45.760182 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:45.782615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Pushing event updated for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:45.786219 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.097s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:47.262387 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:04:48.176170 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42836) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:48.302087 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.317074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:48.320856 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Usage tracker for resource:security_group_rule and tenant:9e7a2cb76f034dc0a1c9d4d8feee5fda is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:48.323785 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:48.329463 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:48.413033 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.483263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.492255 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d2c42c1d-3d2b-44dc-8650-7e652163312c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:48.532863 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=9e7a2cb76f034dc0a1c9d4d8feee5fda&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3556952 May 25 16:04:48.534780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=25715949-a393-4c51-b249-5bb4767bd0ec,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:48.535592 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42846) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:48.538120 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:48.600697 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.637126 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.766559 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.772303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:48.790325 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=25715949-a393-4c51-b249-5bb4767bd0ec,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:48.793522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:48.794647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:48.796203 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/25715949-a393-4c51-b249-5bb4767bd0ec HTTP/1.1" status: 204 len: 168 time: 0.2594194 May 25 16:04:48.799368 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520', 'status': 'completed', 'code': 200} May 25 16:04:48.812760 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-e3bf746b-a66c-4938-b4f7-d962c2d6576e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:48.814601 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-e3bf746b-a66c-4938-b4f7-d962c2d6576e service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 1096 time: 0.1126790 May 25 16:04:48.904055 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7425f8f-c5bb-4c96-8671-ec5a610bebd8 req-c12e60de-c3a1-49c7-9091-cac85638c64a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:48.905683 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e7425f8f-c5bb-4c96-8671-ec5a610bebd8 req-c12e60de-c3a1-49c7-9091-cac85638c64a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1165745 May 25 16:04:49.004789 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e7425f8f-c5bb-4c96-8671-ec5a610bebd8 req-196fd6f7-d05c-4cb1-9f87-687b8920a880 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0627456 May 25 16:04:49.006906 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-9b500043-257b-4098-8d29-fb4bc58983c7 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=08ee0981-245d-4233-9301-2d6fa104b520 HTTP/1.1" status: 200 len: 217 time: 0.1694090 May 25 16:04:49.041272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f79c2c05-cb82-4277-b22d-198afcdbf6ef req-56bcef1a-2d54-4195-94d7-2398d76e6089 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:49.042858 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f79c2c05-cb82-4277-b22d-198afcdbf6ef req-56bcef1a-2d54-4195-94d7-2398d76e6089 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 943 time: 0.1362543 May 25 16:04:49.079025 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-023a4d47-12fa-438f-abf2-634e379d960c req-bb1a8990-4228-4cbc-8092-88d4fa64846c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:49.080461 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-bb1a8990-4228-4cbc-8092-88d4fa64846c service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c HTTP/1.1" status: 200 len: 849 time: 0.0656116 May 25 16:04:49.104028 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f79c2c05-cb82-4277-b22d-198afcdbf6ef req-e1046c5a-74bc-46e3-a4a0-ab22ecef5538 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0527680 May 25 16:04:49.152620 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-00428391-35f3-49da-bf24-c862f45b6181 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=54cf369d-e27b-4d90-a4f4-d729a1eb711e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0529928 May 25 16:04:49.220909 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e0783d2-35d6-4322-a59b-c526c27ca033 req-99dd59e0-d8a2-49be-8d59-a817436cda29 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:49.228487 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e0783d2-35d6-4322-a59b-c526c27ca033 req-99dd59e0-d8a2-49be-8d59-a817436cda29 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.1141996 May 25 16:04:49.296742 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e0783d2-35d6-4322-a59b-c526c27ca033 req-c3236bb3-3902-4947-9ce6-18c51b762462 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0668035 May 25 16:04:49.333953 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-c540cebc-64cd-4dad-aec4-9e696a94d896 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1706753 May 25 16:04:49.472470 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-023a4d47-12fa-438f-abf2-634e379d960c req-f7c2b9cf-9901-428b-872c-80a8cb5db4f7 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1292512 May 25 16:04:49.652310 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42864) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:49.670937 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:49.686293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:49.690305 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Usage tracker for resource:security_group_rule and tenant:be2eae42641646dfb79e65631b78d95b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:49.693581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:49.699873 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:49.817358 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-1fe358d7-76ea-4bef-a764-6a5f6e2243d1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:49.818941 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-1fe358d7-76ea-4bef-a764-6a5f6e2243d1 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 1096 time: 0.1133249 May 25 16:04:49.888728 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:49.992996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:50.004334 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-01a5c54e-ec7d-48da-b9f8-7bac42220be1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:50.021462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-407bb765-19c5-4fe2-9410-a80c9cbef5f9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:50.022906 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-407bb765-19c5-4fe2-9410-a80c9cbef5f9 service neutron] 10.222.0.42 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 910 time: 0.1538479 May 25 16:04:50.111947 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=be2eae42641646dfb79e65631b78d95b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4581544 May 25 16:04:50.114566 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-606c7e85-7148-4989-ae70-635481fe719a service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=08ee0981-245d-4233-9301-2d6fa104b520 HTTP/1.1" status: 200 len: 217 time: 0.0723913 May 25 16:04:50.115049 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 42878) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:50.117361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=be2eec57-ce82-4e0c-8596-8548d855ba20,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:50.120514 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:50.205079 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d9b5ac18-071d-4571-8742-fe142e7f2949 req-e7af79b9-dcd6-4b43-b7d1-ea6491d9b4ce tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:50.206842 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d9b5ac18-071d-4571-8742-fe142e7f2949 req-e7af79b9-dcd6-4b43-b7d1-ea6491d9b4ce tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0948477 May 25 16:04:50.217302 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-01196f7c-60bb-4bbe-89e8-8926424371c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:50.218799 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-01196f7c-60bb-4bbe-89e8-8926424371c6 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c HTTP/1.1" status: 200 len: 849 time: 0.0871501 May 25 16:04:50.278942 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-ad720d58-3ba6-4abe-bf6b-4bf70af2f83d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=54cf369d-e27b-4d90-a4f4-d729a1eb711e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0511003 May 25 16:04:50.283586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:50.308134 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d9b5ac18-071d-4571-8742-fe142e7f2949 req-cc963aad-a33e-4cc9-868c-172b34ce115b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0935717 May 25 16:04:50.349113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b6046996-7f11-4e55-b591-cbe540e2a0f4 req-8574bcbc-014d-48c0-8483-c1738cde8555 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:50.349113 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b6046996-7f11-4e55-b591-cbe540e2a0f4 req-8574bcbc-014d-48c0-8483-c1738cde8555 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 943 time: 0.0977139 May 25 16:04:50.388851 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:50.400007 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b6046996-7f11-4e55-b591-cbe540e2a0f4 req-5715d8cf-40e0-40e8-82b2-432724351001 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0472169 May 25 16:04:50.482425 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-5de9fa50-332e-4179-a767-d8f28ffb0a05 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1892805 May 25 16:04:50.517146 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d21b9b43-dbc2-4304-9f45-cf1477d8c9a0 req-a64d602a-b12a-416a-85f8-68e3fa44839f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:50.518757 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d21b9b43-dbc2-4304-9f45-cf1477d8c9a0 req-a64d602a-b12a-416a-85f8-68e3fa44839f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.0896535 May 25 16:04:50.573316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d21b9b43-dbc2-4304-9f45-cf1477d8c9a0 req-1eed9c85-be1f-4682-8db9-48c0cc8f55f5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0479705 May 25 16:04:50.577626 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:50.583835 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-178dbb17-d6d4-4688-b2ce-54e84bc7fa0c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:50.606134 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=be2eec57-ce82-4e0c-8596-8548d855ba20,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:50.608549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:50.620938 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:50.622111 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/be2eec57-ce82-4e0c-8596-8548d855ba20 HTTP/1.1" status: 204 len: 168 time: 0.5060191 May 25 16:04:50.660827 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ed77d211-cae6-4ed2-b5ff-614f83b28601 req-2f2b05c3-4019-49ce-83a9-26a55a43186f service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1671436 May 25 16:04:51.529489 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4048c5e-4787-4b4a-af35-18c6d7cac6c1 req-3ea6c51e-6fd5-421a-8630-2ba8dfdb9003 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:51.531072 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d4048c5e-4787-4b4a-af35-18c6d7cac6c1 req-3ea6c51e-6fd5-421a-8630-2ba8dfdb9003 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1061833 May 25 16:04:51.629182 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d4048c5e-4787-4b4a-af35-18c6d7cac6c1 req-a0f20c9e-96cb-4678-a490-3f85a7b140d6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663574 May 25 16:04:51.669363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fcb7189-3396-4f7f-8cb2-d759d1d9c483 req-45737614-ce08-44bb-8496-68f36eef681e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:51.671229 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4fcb7189-3396-4f7f-8cb2-d759d1d9c483 req-45737614-ce08-44bb-8496-68f36eef681e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 943 time: 0.1392279 May 25 16:04:51.731787 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4fcb7189-3396-4f7f-8cb2-d759d1d9c483 req-7525cc1f-d0ee-44ea-9f24-7c4bab18b63d tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0520372 May 25 16:04:51.835122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d11a9d93-cc22-4787-b8a5-cbcad049695f req-718caab1-42e7-402e-981a-df4fba7f3178 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:51.836867 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d11a9d93-cc22-4787-b8a5-cbcad049695f req-718caab1-42e7-402e-981a-df4fba7f3178 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 943 time: 0.1001513 May 25 16:04:51.886020 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d11a9d93-cc22-4787-b8a5-cbcad049695f req-557b4ade-d07a-40a0-86bb-330a07d51565 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0432119 May 25 16:04:53.656608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42986) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:53.784248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:53.800910 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:53.805336 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Usage tracker for resource:security_group_rule and tenant:4279555563114318aa546b9e519f030c is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:53.808265 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:53.814045 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:53.913418 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.002998 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.013103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-eac3f770-b93c-487a-8a7b-cb57495442d5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:54.053994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f96bd200-a1c7-4b61-90fd-38d3691d7b90,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:54.058736 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=4279555563114318aa546b9e519f030c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4010901 May 25 16:04:54.059744 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:54.062964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 42996) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:54.122738 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.150031 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.260518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.264420 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2fa8c176-ed09-44cb-a46e-2fedc4a930f4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:54.279163 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f96bd200-a1c7-4b61-90fd-38d3691d7b90,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:54.281635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:54.295795 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.296947 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f96bd200-a1c7-4b61-90fd-38d3691d7b90 HTTP/1.1" status: 204 len: 168 time: 0.2330320 May 25 16:04:54.508892 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2206a32c-5500-4208-9123-7aab17fc53c7 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:04:54.683286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2206a32c-5500-4208-9123-7aab17fc53c7,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:54.697027 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '2206a32c-5500-4208-9123-7aab17fc53c7', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'port_id': '2206a32c-5500-4208-9123-7aab17fc53c7', 'mac_address': 'fa:16:3e:b4:7e:fa', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 948, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b19f7657-6038-4479-b9ec-9e4e92c69bb0', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:04:54.767818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.792763 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 2206a32c-5500-4208-9123-7aab17fc53c7 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:54.812065 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2206a32c-5500-4208-9123-7aab17fc53c7,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:54.911507 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:54.913443 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:54.990449 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:54.993797 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:55.155193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43008) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:55.172897 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.187702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:55.191586 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Usage tracker for resource:security_group_rule and tenant:f2444125909649b4bb4a39dacf209bed is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:55.194331 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:55.200550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:55.281841 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.419600 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.427223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:55.468119 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f2444125909649b4bb4a39dacf209bed&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3118024 May 25 16:04:55.469068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2c07802f-046e-4208-a4fc-dadf6a6cd4d0,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:55.471657 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 43012) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:55.472413 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:55.532493 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.559021 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.684125 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.689991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a2fd5669-e3a8-481c-8bee-729a2ea95e69 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:55.704830 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2c07802f-046e-4208-a4fc-dadf6a6cd4d0,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:55.709624 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:55.710252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:55.714673 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2c07802f-046e-4208-a4fc-dadf6a6cd4d0 HTTP/1.1" status: 204 len: 168 time: 0.2419355 May 25 16:04:55.736394 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 08ee0981-245d-4233-9301-2d6fa104b520 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:04:55.867635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f23f9422-5035-4dff-b188-ab70daf6af1f req-bf3174aa-7421-4e78-9fdd-3c8f341354dd tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:55.875493 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f23f9422-5035-4dff-b188-ab70daf6af1f req-bf3174aa-7421-4e78-9fdd-3c8f341354dd tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1014037 May 25 16:04:55.957094 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=08ee0981-245d-4233-9301-2d6fa104b520,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:55.967716 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f23f9422-5035-4dff-b188-ab70daf6af1f req-b67ea735-ff6f-4831-9a9d-4fce56812908 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0911930 May 25 16:04:55.976208 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '08ee0981-245d-4233-9301-2d6fa104b520', 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e', 'port_id': '08ee0981-245d-4233-9301-2d6fa104b520', 'mac_address': 'fa:16:3e:47:db:16', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 918, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4fa131e2-5aa0-4f62-bf74-2a67a79ad81c', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:04:56.009659 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-21a1237a-2b44-48b2-9abf-8c9a5bf174fd req-dc575cde-2caf-4f8a-b0eb-7cee8cc61078 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:56.020140 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-21a1237a-2b44-48b2-9abf-8c9a5bf174fd req-dc575cde-2caf-4f8a-b0eb-7cee8cc61078 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 943 time: 0.1414530 May 25 16:04:56.038410 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:56.067928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 08ee0981-245d-4233-9301-2d6fa104b520 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:04:56.071448 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-21a1237a-2b44-48b2-9abf-8c9a5bf174fd req-6d11b9dd-ca67-48bd-97f9-7c70737cdef9 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0499322 May 25 16:04:56.090085 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=08ee0981-245d-4233-9301-2d6fa104b520,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:56.175109 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:56.178157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:56.267622 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['2206a32c-5500-4208-9123-7aab17fc53c7'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:56.279139 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5f7a28d5-c181-4738-b6da-99eaf7cf6696 req-a7a26750-659f-40e3-86f0-eb17f3362b21 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:56.280857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5f7a28d5-c181-4738-b6da-99eaf7cf6696 req-a7a26750-659f-40e3-86f0-eb17f3362b21 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 944 time: 0.1801674 May 25 16:04:56.321351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:56.324881 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.147s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:56.344641 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5f7a28d5-c181-4738-b6da-99eaf7cf6696 req-eb2a3dfc-16e9-4c27-a9bf-dcb966c3473a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0571239 May 25 16:04:56.737323 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['08ee0981-245d-4233-9301-2d6fa104b520'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:04:57.061178 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 43048) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:57.209463 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f98ca37b-6d0d-4043-b5d7-a1e94b4dfab5 req-7bb8d8b4-be8f-40e3-8428-2b751e83539c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:57.211171 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-764100798-network', 'tenant_id': '6055d0e4eebd4015979060d95f3f0f8c'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:57.216037 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f98ca37b-6d0d-4043-b5d7-a1e94b4dfab5 req-7bb8d8b4-be8f-40e3-8428-2b751e83539c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1151180 May 25 16:04:57.216693 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:57.222776 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Usage tracker for resource:network and tenant:6055d0e4eebd4015979060d95f3f0f8c is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:57.227458 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:57.234325 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:57.321306 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f98ca37b-6d0d-4043-b5d7-a1e94b4dfab5 req-6b453c92-8f48-49bc-a8eb-e298057aeb0f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1016378 May 25 16:04:57.323490 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Made reservation on behalf of 6055d0e4eebd4015979060d95f3f0f8c for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:57.326229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.345828 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.372024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:57.376574 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Usage tracker for resource:security_group_rule and tenant:6055d0e4eebd4015979060d95f3f0f8c is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:57.382997 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce617818-4c8b-4c1f-beb7-82b6c8636299 req-29be5775-1233-4cb0-8568-33b78c97e8a2 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:57.383282 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:57.383375 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ce617818-4c8b-4c1f-beb7-82b6c8636299 req-29be5775-1233-4cb0-8568-33b78c97e8a2 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 944 time: 0.1450336 May 25 16:04:57.387150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:57.465300 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ce617818-4c8b-4c1f-beb7-82b6c8636299 req-bb426af4-ad0a-4c8a-b631-ebfb6f8ca904 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0754452 May 25 16:04:57.524799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.600974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-79fcf93c-fa73-4c08-b383-c8573f00df9f req-ff78ee0f-1e64-4b5e-8039-5b066a0362ba tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:57.603032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.603450 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-79fcf93c-fa73-4c08-b383-c8573f00df9f req-ff78ee0f-1e64-4b5e-8039-5b066a0362ba tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 944 time: 0.0869958 May 25 16:04:57.607761 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e9b2b39a-30ae-46b1-937a-76f522ff4b00 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:57.634183 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] vlan segment allocate from pool success with {'vlan_id': 925, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:04:57.650192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d60c2e4c-2be2-42d4-99d5-8e3616307a3b,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:57.654509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:57.655521 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.655860 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Added segment bbabde65-843a-4b69-8726-c5a9d990a020 of type vlan for network db850d4d-e971-4d97-b059-7087190eebdd May 25 16:04:57.671516 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-79fcf93c-fa73-4c08-b383-c8573f00df9f req-243fc080-d5be-4ccd-9abc-8a96abccead5 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0617971 May 25 16:04:57.685251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.716470 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2206a32c-5500-4208-9123-7aab17fc53c7 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:04:57.831383 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.832822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:57.833717 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:57.842213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:57.847777 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43072) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:57.865669 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 2206a32c-5500-4208-9123-7aab17fc53c7 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:04:57.871233 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 2206a32c-5500-4208-9123-7aab17fc53c7 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:04:57.871824 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.004199 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.010146 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.030679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:58.035229 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Usage tracker for resource:security_group_rule and tenant:b035abadef5d4cd3b4a785b6f3255614 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:58.037298 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:04:58.039231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:58.046490 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:58.049538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2206a32c-5500-4208-9123-7aab17fc53c7,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:04:58.134263 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.136224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:58.147999 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.170864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Pushing event updated for resources: {'Network': ['ID=db850d4d-e971-4d97-b059-7087190eebdd,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:58.177019 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7', 'code': 200} May 25 16:04:58.178424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.340s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:58.192601 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network db850d4d-e971-4d97-b059-7087190eebdd details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:04:58.226583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.236511 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-68a21efe-b4ff-474e-b102-a5c144915577 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:58.261220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43084) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:58.261847 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.1980274 May 25 16:04:58.282081 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Request body: {'subnet': {'network_id': 'db850d4d-e971-4d97-b059-7087190eebdd', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-764100798-subnet', 'tenant_id': '6055d0e4eebd4015979060d95f3f0f8c', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:04:58.290777 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b801b99f-0f78-4596-a023-8e0efbec51ef,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:58.294006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:58.295390 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:04:58.302251 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=b035abadef5d4cd3b4a785b6f3255614&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4509032 May 25 16:04:58.302615 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:58.303329 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43088) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:04:58.304992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Usage tracker for resource:subnet and tenant:6055d0e4eebd4015979060d95f3f0f8c is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:04:58.307216 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.171s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:58.308846 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:04:58.318136 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:04:58.384629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.415029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.476932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Made reservation on behalf of 6055d0e4eebd4015979060d95f3f0f8c for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:04:58.479229 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.697468 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d03dc62-37df-418f-9a52-cc10b96dd16a req-11828f18-f034-4ff7-8b2b-bf4b24c344c1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:58.699190 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2d03dc62-37df-418f-9a52-cc10b96dd16a req-11828f18-f034-4ff7-8b2b-bf4b24c344c1 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 1098 time: 0.0940974 May 25 16:04:58.744752 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-46dc9bf4-88fb-4c61-b872-c3f7016d3d29 req-e11c457e-6ac1-4847-8827-693dcfbac216 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:58.746283 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-46dc9bf4-88fb-4c61-b872-c3f7016d3d29 req-e11c457e-6ac1-4847-8827-693dcfbac216 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.3106763 May 25 16:04:58.762749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52facdef-991e-4c4f-9725-d0144674f22f req-4a4e25ed-46a8-4214-bd76-80f424970b3a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:58.764101 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2d03dc62-37df-418f-9a52-cc10b96dd16a req-fc3df6fa-1c36-47fd-b6b9-4be3928c7fad service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0513697 May 25 16:04:58.764309 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-52facdef-991e-4c4f-9725-d0144674f22f req-4a4e25ed-46a8-4214-bd76-80f424970b3a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 944 time: 0.1516924 May 25 16:04:58.814158 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.820179 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4224686b-335d-4ff2-a18a-90cae69c1a01 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:58.854326 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b801b99f-0f78-4596-a023-8e0efbec51ef,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:58.857653 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.037s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:58.883516 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-52facdef-991e-4c4f-9725-d0144674f22f req-85bc2bda-2e8f-40a9-bc45-16f3e6efcfde tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1022444 May 25 16:04:58.886247 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-46dc9bf4-88fb-4c61-b872-c3f7016d3d29 req-6529d23d-4d58-432a-a84e-280e851312e6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1209738 May 25 16:04:58.938377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:58.942574 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/b801b99f-0f78-4596-a023-8e0efbec51ef HTTP/1.1" status: 204 len: 168 time: 0.6231010 May 25 16:04:59.089320 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-64225f0b-7910-46bb-98f0-41a2dd5967d7 req-a89454bb-f634-46b0-bd62-e1085df74d69 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:59.090911 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-64225f0b-7910-46bb-98f0-41a2dd5967d7 req-a89454bb-f634-46b0-bd62-e1085df74d69 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.0906944 May 25 16:04:59.140614 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-64225f0b-7910-46bb-98f0-41a2dd5967d7 req-3312aef6-e432-4931-8355-febc11dcfeba tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427828 May 25 16:04:59.348606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:59.352716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:04:59.430044 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Pushing event updated for resources: {'Subnet': ['ID=1c61f546-0daa-4a78-b2f0-d09f03693bd8,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:04:59.434218 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:04:59.438928 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Checking segment: {'id': 'bbabde65-843a-4b69-8726-c5a9d990a020', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 925, 'network_id': 'db850d4d-e971-4d97-b059-7087190eebdd'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:04:59.439287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Network hostable DHCP agents. Network: db850d4d-e971-4d97-b059-7087190eebdd, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:04:59.648871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Network db850d4d-e971-4d97-b059-7087190eebdd is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:04:59.717158 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 08ee0981-245d-4233-9301-2d6fa104b520 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:04:59.751900 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, db850d4d-e971-4d97-b059-7087190eebdd) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:04:59.768002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:04:59.769069 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:04:59.773500 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5b94cc33-d0bc-4503-a6bf-55bd4af0bb62 None None] Network db850d4d-e971-4d97-b059-7087190eebdd requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:04:59.872708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-4793ad18-1d49-46de-b997-def0cc440417 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 1.6038897 May 25 16:05:00.001233 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 08ee0981-245d-4233-9301-2d6fa104b520 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:05:00.005739 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-94afdbb9-ac10-4b02-9050-99cf6b6e3368 None None] Network db850d4d-e971-4d97-b059-7087190eebdd requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:05:00.118192 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 08ee0981-245d-4233-9301-2d6fa104b520 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:05:00.118570 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:00.211464 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68cb60c1-589b-4d8f-94b8-5f7a2d442d24 req-4bdfa088-b447-4dba-a110-755b00169fd6 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:00.216640 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68cb60c1-589b-4d8f-94b8-5f7a2d442d24 req-4bdfa088-b447-4dba-a110-755b00169fd6 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 944 time: 0.1871762 May 25 16:05:00.273614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bad9422e-388e-4a3c-b605-0b87b3ae95eb req-e1a19891-3f93-42a7-96ff-4671c7dbe14f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:00.280763 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bad9422e-388e-4a3c-b605-0b87b3ae95eb req-e1a19891-3f93-42a7-96ff-4671c7dbe14f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1856902 May 25 16:05:00.358112 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68cb60c1-589b-4d8f-94b8-5f7a2d442d24 req-7f0d5194-399c-426b-ab40-40426ea66985 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1295028 May 25 16:05:00.379725 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:00.395061 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bad9422e-388e-4a3c-b605-0b87b3ae95eb req-cf6d11e1-d5c0-49e3-8c40-bd420c52c1b8 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1117518 May 25 16:05:00.434545 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=08ee0981-245d-4233-9301-2d6fa104b520,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:00.436388 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:00.567030 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:00.570246 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:00.606332 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-294bbf07-b1ef-43a1-97b8-a1ccf3fb0e2c req-2ee58f1f-5443-4d35-9d6d-f5f636f92e73 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:00.607949 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-294bbf07-b1ef-43a1-97b8-a1ccf3fb0e2c req-2ee58f1f-5443-4d35-9d6d-f5f636f92e73 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.1060958 May 25 16:05:00.664028 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:00.677140 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:00.679096 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-294bbf07-b1ef-43a1-97b8-a1ccf3fb0e2c req-dd025c0d-a58c-4391-9430-20d73dde4d5f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0629044 May 25 16:05:00.737451 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520', 'code': 200} May 25 16:05:00.928814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43204) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:00.935702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Request body: {'network': {'name': 'tempest-VolumesActionsTest-1520780052-network', 'tenant_id': '0f31b747aa294350b3392970ce891fe3'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:00.940642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:00.946195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Usage tracker for resource:network and tenant:0f31b747aa294350b3392970ce891fe3 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:00.949506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:00.956487 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:01.037450 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Made reservation on behalf of 0f31b747aa294350b3392970ce891fe3 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:01.041144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.136961 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.152418 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:01.156698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Usage tracker for resource:security_group_rule and tenant:0f31b747aa294350b3392970ce891fe3 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:01.159874 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:01.167127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:01.257027 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.362988 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.366287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5b234f8d-7c11-44cb-ab88-9ad1ed63675f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:01.396881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] vlan segment allocate from pool success with {'vlan_id': 905, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:05:01.406473 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a0327111-6609-4f1e-a645-1f604163df8b,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:01.409825 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:01.416206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.416660 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Added segment 55bc8fb5-7948-478b-9d7a-e4b5d9e1ac5d of type vlan for network cb9233b3-2734-4a6a-9749-ee4b98a4a54a May 25 16:05:01.449206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.625645 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.627127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:01.627765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:01.631628 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:01.738824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Pushing event updated for resources: {'Network': ['ID=cb9233b3-2734-4a6a-9749-ee4b98a4a54a,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:01.741764 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:01.749337 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network cb9233b3-2734-4a6a-9749-ee4b98a4a54a details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:05:01.815802 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 0.8852057 May 25 16:05:01.818530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43218) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:01.824550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Request body: {'subnet': {'network_id': 'cb9233b3-2734-4a6a-9749-ee4b98a4a54a', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesActionsTest-1520780052-subnet', 'tenant_id': '0f31b747aa294350b3392970ce891fe3', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:01.834549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:01.840414 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Usage tracker for resource:subnet and tenant:0f31b747aa294350b3392970ce891fe3 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:01.845619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:01.857859 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:01.934115 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Made reservation on behalf of 0f31b747aa294350b3392970ce891fe3 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:01.936496 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:02.425528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:02.430315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1b17e03e-4bf2-4eee-8f09-0be9a9655be4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:02.503707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Pushing event updated for resources: {'Subnet': ['ID=02869cfb-fcb1-40ce-b14b-a2b3200f595b,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:02.508296 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:02.515899 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Checking segment: {'id': '55bc8fb5-7948-478b-9d7a-e4b5d9e1ac5d', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 905, 'network_id': 'cb9233b3-2734-4a6a-9749-ee4b98a4a54a'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:05:02.516256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Network hostable DHCP agents. Network: cb9233b3-2734-4a6a-9749-ee4b98a4a54a, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:05:02.591587 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Network cb9233b3-2734-4a6a-9749-ee4b98a4a54a is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:05:02.700254 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, cb9233b3-2734-4a6a-9749-ee4b98a4a54a) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:02.709102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-37ac2203-5c08-4cb2-81d0-d468bdc86775 None None] Network cb9233b3-2734-4a6a-9749-ee4b98a4a54a requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:05:02.709330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:02.710412 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:02.849518 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ebaf4817-2fd4-465e-b97e-76a3dbc3ab4b admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 846 time: 1.0297136 May 25 16:05:02.921927 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f39b5905-8619-4054-b2d0-efc53656def9 None None] Network cb9233b3-2734-4a6a-9749-ee4b98a4a54a requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:05:04.106478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94496a1c-a24b-4b58-ac32-eef43993abdf req-c8319752-237a-41f9-81c0-94561c5df52e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:04.108044 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94496a1c-a24b-4b58-ac32-eef43993abdf req-c8319752-237a-41f9-81c0-94561c5df52e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1690509 May 25 16:05:04.166972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8ee2676-ba72-4226-83c0-ff53986259c0 req-9a62abcc-8a71-415b-91de-3c48194eae57 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:04.166972 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8ee2676-ba72-4226-83c0-ff53986259c0 req-9a62abcc-8a71-415b-91de-3c48194eae57 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1504321 May 25 16:05:04.205161 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94496a1c-a24b-4b58-ac32-eef43993abdf req-8513a18a-eb77-4080-bccb-984d7fca4534 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0833833 May 25 16:05:04.238277 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8ee2676-ba72-4226-83c0-ff53986259c0 req-8ab2a4b4-6364-43dd-be89-995669bb390d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0649645 May 25 16:05:04.362147 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-452bc963-a193-4fa7-8e07-7d11749199b8 req-4ccbc64e-e8ce-4d0c-b9bf-3e4cd1e07cb1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:04.364306 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-452bc963-a193-4fa7-8e07-7d11749199b8 req-4ccbc64e-e8ce-4d0c-b9bf-3e4cd1e07cb1 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 1098 time: 0.1131740 May 25 16:05:04.412372 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-452bc963-a193-4fa7-8e07-7d11749199b8 req-8967c397-651d-4356-8bf5-18255b81c8ff service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0405760 May 25 16:05:04.450072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-975bec3e-976b-4a36-84d8-d4f24b4a0cc4 req-5e8ddaa4-b611-4b92-ae0a-0c19b65594e2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:04.451768 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-975bec3e-976b-4a36-84d8-d4f24b4a0cc4 req-5e8ddaa4-b611-4b92-ae0a-0c19b65594e2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.0927117 May 25 16:05:04.500740 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-975bec3e-976b-4a36-84d8-d4f24b4a0cc4 req-4f37f8a9-f819-461e-8d25-ed3e3a3927e1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0430295 May 25 16:05:05.565352 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9adab553-4e68-49dc-8936-774943a82309 req-9cbe464f-682f-4dba-b08b-15bbbfad2ead tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:05.567046 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9adab553-4e68-49dc-8936-774943a82309 req-9cbe464f-682f-4dba-b08b-15bbbfad2ead tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1606972 May 25 16:05:05.579737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5d5319f-6860-4ca8-a40f-57a16a67906b req-3d43180c-647a-4813-b908-29ba22ee56ca tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:05.581811 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5d5319f-6860-4ca8-a40f-57a16a67906b req-3d43180c-647a-4813-b908-29ba22ee56ca tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1387374 May 25 16:05:05.665756 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9adab553-4e68-49dc-8936-774943a82309 req-417dbd2c-82d5-4c57-b086-a5fd60890070 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0782564 May 25 16:05:05.680646 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5d5319f-6860-4ca8-a40f-57a16a67906b req-6f8a6574-732d-4b40-ae9a-8d923cac7e96 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0700533 May 25 16:05:05.854274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d733edeb-560e-49f2-892c-7db98b218215 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:06.952883 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd2db872-ec50-4fed-9253-f628535af5d3 req-047d73a2-df25-4270-a87a-a3a3001c8601 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:06.954285 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd2db872-ec50-4fed-9253-f628535af5d3 req-047d73a2-df25-4270-a87a-a3a3001c8601 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1275461 May 25 16:05:06.961887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4faca7ff-70cc-4410-88c7-0b941a2772f0 req-d60980d0-4986-40ba-8909-64c6c369e112 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:06.963332 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4faca7ff-70cc-4410-88c7-0b941a2772f0 req-d60980d0-4986-40ba-8909-64c6c369e112 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1709824 May 25 16:05:07.037508 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd2db872-ec50-4fed-9253-f628535af5d3 req-13074285-9f3e-4e38-827d-a8812b912e0d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0732069 May 25 16:05:07.040259 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4faca7ff-70cc-4410-88c7-0b941a2772f0 req-98974f92-0a37-4751-a808-15784a1b944d tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0610530 May 25 16:05:08.134282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:08.287029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b349906d-a918-4854-a2dd-d0b89d55aefd req-276a5361-efb8-405d-93ae-65d01759ded6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:08.290349 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b349906d-a918-4854-a2dd-d0b89d55aefd req-276a5361-efb8-405d-93ae-65d01759ded6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1361032 May 25 16:05:08.352257 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b349906d-a918-4854-a2dd-d0b89d55aefd req-903c9f2c-32bd-4a62-86ec-faf1b82aaea5 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0536013 May 25 16:05:08.448873 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8249aba8-736e-4512-8f94-c3ebbaa624d2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:08.609697 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d66b467e-964e-4dda-aa50-1462b3604fe1 req-2592e692-ef4b-4d56-80d6-a8719be4ff7d tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:08.611277 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d66b467e-964e-4dda-aa50-1462b3604fe1 req-2592e692-ef4b-4d56-80d6-a8719be4ff7d tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1281767 May 25 16:05:08.658666 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d66b467e-964e-4dda-aa50-1462b3604fe1 req-ad84e39d-394c-4ba7-8e23-5d80b1175c72 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0408423 May 25 16:05:09.059534 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:09.320932 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-3c62c8b3-f671-45d0-8065-80740324feb8 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0735505 May 25 16:05:09.459754 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-6f646205-b4da-4422-8678-22ebf4dc0d16 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:09.462059 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-6f646205-b4da-4422-8678-22ebf4dc0d16 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 1099 time: 0.0872109 May 25 16:05:09.669691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69f93cac-a334-44d8-9d61-e8e7c21c1fb5 req-c29ffbf8-7931-415f-8b25-7f28a69e9b30 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:09.671212 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-69f93cac-a334-44d8-9d61-e8e7c21c1fb5 req-c29ffbf8-7931-415f-8b25-7f28a69e9b30 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1926284 May 25 16:05:09.672006 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-6641f60d-8223-4889-9c8b-80be52b324f9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:09.673436 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-6641f60d-8223-4889-9c8b-80be52b324f9 service neutron] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 915 time: 0.1604559 May 25 16:05:09.718175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-69f93cac-a334-44d8-9d61-e8e7c21c1fb5 req-3f03bffe-a86e-47c0-90af-d1c22430f27c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0413005 May 25 16:05:09.754729 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-e9372796-41f5-4455-b2ae-8a66d6168b2d service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc HTTP/1.1" status: 200 len: 217 time: 0.0725524 May 25 16:05:09.830717 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-da324e05-e348-4b88-8602-2cd2d763800b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:09.832337 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-da324e05-e348-4b88-8602-2cd2d763800b service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.0614245 May 25 16:05:09.889546 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-35b5c977-9378-4975-8e88-9d606e49e3c9 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0494153 May 25 16:05:10.094192 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-eb4e6100-e4f6-4205-9880-301a472b0ca6 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1965704 May 25 16:05:10.240116 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3b230e7e-96d3-484b-a7c8-fb22180fc74b req-4810a80e-4d22-4950-b85a-7d553370b2d3 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1377888 May 25 16:05:10.453903 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:10.916004 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4d0d7c50-204a-4e96-8b1c-a59ad79a9396 req-64cdfdca-34e8-4c69-b777-25c23e78faec tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:10.917469 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4d0d7c50-204a-4e96-8b1c-a59ad79a9396 req-64cdfdca-34e8-4c69-b777-25c23e78faec tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0827663 May 25 16:05:10.970517 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4d0d7c50-204a-4e96-8b1c-a59ad79a9396 req-4a77ca86-3155-4bc3-86ec-9215ea3ed099 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0470018 May 25 16:05:11.174115 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:11.634748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-80b1b5c9-da15-4a34-a3fb-12e16b53dba8 req-f4fcd452-3bc4-467a-a7c5-afb66c805bdc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:11.636232 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-80b1b5c9-da15-4a34-a3fb-12e16b53dba8 req-f4fcd452-3bc4-467a-a7c5-afb66c805bdc service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.0802743 May 25 16:05:11.681119 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-80b1b5c9-da15-4a34-a3fb-12e16b53dba8 req-4f9daece-9281-402b-ba2f-cba2c8cddd84 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0368667 May 25 16:05:12.091321 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 43430) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:12.251852 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.255397 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d319ea3-60b4-4660-ad00-87100c089fc1 req-d8ff7221-4ba5-4495-90f1-36411972e001 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:12.257576 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d319ea3-60b4-4660-ad00-87100c089fc1 req-d8ff7221-4ba5-4495-90f1-36411972e001 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1297863 May 25 16:05:12.266938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2cda8a22-b85f-4074-ab20-b5a293d3b62d req-89f15f70-1cd0-4c38-863e-c43d376d905d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:12.268480 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2cda8a22-b85f-4074-ab20-b5a293d3b62d req-89f15f70-1cd0-4c38-863e-c43d376d905d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1771193 May 25 16:05:12.278213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:12.284078 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Usage tracker for resource:security_group_rule and tenant:6dd5ef2d4a6d4f02bc26406587db6af2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:12.287605 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:12.302044 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:12.334435 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d319ea3-60b4-4660-ad00-87100c089fc1 req-e31c7789-b003-4260-896d-561295345f17 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0649157 May 25 16:05:12.364173 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2cda8a22-b85f-4074-ab20-b5a293d3b62d req-da2869f8-5cd5-441c-a3ee-26a7067dedd6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0745692 May 25 16:05:12.415626 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.522590 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.530488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-63a747ee-060d-4532-bf73-53d90a468d4e admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:12.584510 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=6dd5ef2d4a6d4f02bc26406587db6af2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4904788 May 25 16:05:12.585509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9024c193-c530-4512-976d-4460afb1f75d,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:12.586047 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43444) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:12.597262 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.597884 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.598399 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.598937 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.599462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.599981 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.600334 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.601308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.601660 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.602273 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.602630 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.603016 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.608450 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:12.661871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.696668 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.838650 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.842682 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-19657c74-405b-42d0-8504-b082306ef614 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:12.856377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9024c193-c530-4512-976d-4460afb1f75d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:12.869372 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.869372 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.869894 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.870243 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.870576 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.870928 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.871262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.871594 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.871929 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.872319 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.872654 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.873206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:12.876933 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:12.878546 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:12.879709 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/9024c193-c530-4512-976d-4460afb1f75d HTTP/1.1" status: 204 len: 168 time: 0.2926042 May 25 16:05:13.612261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b467291e-f535-4dca-8b46-bb5aa74cc8df req-6b23e39f-6335-49d7-8468-1fa2a8b88d21 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:13.613802 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b467291e-f535-4dca-8b46-bb5aa74cc8df req-6b23e39f-6335-49d7-8468-1fa2a8b88d21 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1445107 May 25 16:05:13.626861 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ef7fe23-5fd9-412f-bec8-9dddf232184e req-4cee94dc-6c30-49be-b9f8-808b2be211f3 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:13.628423 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ef7fe23-5fd9-412f-bec8-9dddf232184e req-4cee94dc-6c30-49be-b9f8-808b2be211f3 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1272202 May 25 16:05:13.710511 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b467291e-f535-4dca-8b46-bb5aa74cc8df req-55c592d8-f4e7-4a32-8224-c3bcbb229635 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0805316 May 25 16:05:13.716204 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ef7fe23-5fd9-412f-bec8-9dddf232184e req-c11ff227-c81b-4f7e-ae1a-f79560112955 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0662696 May 25 16:05:13.792099 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43468) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:13.815095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:13.832805 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:13.837132 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Usage tracker for resource:security_group_rule and tenant:ef87435ce8dd4215b5e8de454e40a69f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:13.840097 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:13.846244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:13.942540 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.088311 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.104308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:14.144499 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=ef87435ce8dd4215b5e8de454e40a69f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3512022 May 25 16:05:14.146241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e96490dd-76ca-4415-85cd-64f6a94480aa,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:14.148035 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 43476) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:14.149341 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:14.214034 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.243746 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.373659 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.377537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7261ed2b-786a-413c-9847-b9f425dbd919 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:14.390833 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e96490dd-76ca-4415-85cd-64f6a94480aa,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:14.394522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:14.402619 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:14.404143 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/e96490dd-76ca-4415-85cd-64f6a94480aa HTTP/1.1" status: 204 len: 168 time: 0.2542970 May 25 16:05:14.983953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7c2f50db-2cbc-41e0-8ce1-20a363abf542 req-a19cce68-4b0b-451a-bf2a-5237d0a786aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:14.987470 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7c2f50db-2cbc-41e0-8ce1-20a363abf542 req-a19cce68-4b0b-451a-bf2a-5237d0a786aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1113129 May 25 16:05:15.072659 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7c2f50db-2cbc-41e0-8ce1-20a363abf542 req-3c37ed63-bd96-484a-9c35-9e9f6a8e108b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0803106 May 25 16:05:15.135476 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-743f1c60-acdf-454a-bb2d-40ba85feae7c req-bf4e25e9-46bc-4f30-80cb-9cbf7a0c8088 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:15.137069 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-743f1c60-acdf-454a-bb2d-40ba85feae7c req-bf4e25e9-46bc-4f30-80cb-9cbf7a0c8088 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.1051035 May 25 16:05:15.185549 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-743f1c60-acdf-454a-bb2d-40ba85feae7c req-d18bbb04-cf4a-4034-8176-8ab716a8d020 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420573 May 25 16:05:15.784029 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-12f3834e-365d-499e-9662-0eb2440edb54 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:15.784954 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-12f3834e-365d-499e-9662-0eb2440edb54 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Synchronizing usage tracker for tenant:9e6e907c5841468a85bc59f21b26372d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:05:15.876422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-12f3834e-365d-499e-9662-0eb2440edb54 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Unset dirty status for tenant:9e6e907c5841468a85bc59f21b26372d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:05:15.877748 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-12f3834e-365d-499e-9662-0eb2440edb54 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.2106242 May 25 16:05:15.926016 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5c8a1e7d-e1ff-44c2-bd4e-1d827b159a65 req-97d66eca-283e-4452-b9f4-2491986d5a63 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:15.927220 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5c8a1e7d-e1ff-44c2-bd4e-1d827b159a65 req-97d66eca-283e-4452-b9f4-2491986d5a63 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2262349 May 25 16:05:15.935071 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-03e487ec-823f-45bc-beea-2cc0baac97df service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:15.936706 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-03e487ec-823f-45bc-beea-2cc0baac97df service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.1159666 May 25 16:05:16.080800 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Deleting port 08ee0981-245d-4233-9301-2d6fa104b520 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:05:16.081728 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:16.082605 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5c8a1e7d-e1ff-44c2-bd4e-1d827b159a65 req-4b7783bd-b00e-46ee-a4cb-a5f6ef5012d1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1444297 May 25 16:05:16.135716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-8afcde6b-1e93-4e70-959b-2f925e974fab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:16.137134 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-8afcde6b-1e93-4e70-959b-2f925e974fab service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.1805062 May 25 16:05:16.254232 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-b88da432-5f30-4b58-9e16-92cdb35bdde0 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.0952127 May 25 16:05:16.270733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] For port 08ee0981-245d-4233-9301-2d6fa104b520, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=08ee0981-245d-4233-9301-2d6fa104b520,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:16.271121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:16.389673 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-fab09adc-4c8b-41c8-859e-ecdb79ad3c6a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:16.391382 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-fab09adc-4c8b-41c8-859e-ecdb79ad3c6a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.1247220 May 25 16:05:16.451504 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-d7d96019-58fc-44f5-9578-5ebfbcbea9f3 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0483625 May 25 16:05:16.518694 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Calling delete_port for 08ee0981-245d-4233-9301-2d6fa104b520 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:05:16.652441 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f12742b6-82aa-4161-b954-9dad7dfe4734 req-a93a8a79-a3c4-4fcf-96b0-e774b225e0b1 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:16.654085 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f12742b6-82aa-4161-b954-9dad7dfe4734 req-a93a8a79-a3c4-4fcf-96b0-e774b225e0b1 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 945 time: 0.3300056 May 25 16:05:16.737842 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f12742b6-82aa-4161-b954-9dad7dfe4734 req-f2addd6a-a8dd-4d0a-a53a-f6c4180aa80d tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0753329 May 25 16:05:16.739749 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-20bc77ab-9fde-42d8-be90-30b01b0e7775 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2830250 May 25 16:05:16.745251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:16.759579 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:16.788521 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['08ee0981-245d-4233-9301-2d6fa104b520'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:05:16.818897 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Pushing event deleted for resources: {'Port': ['ID=08ee0981-245d-4233-9301-2d6fa104b520,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:16.832606 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:16.920795 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c24ceed5-719a-4e5e-b5c6-044581d4f682 req-cabe79aa-902b-41e9-a2af-72544c37e9be service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1643238 May 25 16:05:16.947882 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] No DVR routers for this DVR port 08ee0981-245d-4233-9301-2d6fa104b520 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:05:16.950757 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:16.955622 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'name': 'network-vif-deleted', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:17.009810 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'a7885d47-fd9e-463a-be96-bdd6c5b6988f', 'name': 'network-vif-deleted', 'tag': '08ee0981-245d-4233-9301-2d6fa104b520', 'status': 'completed', 'code': 200} May 25 16:05:17.050547 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c778281b-1f88-4a83-b7f6-c928bad3643b req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "DELETE /v2.0/ports/08ee0981-245d-4233-9301-2d6fa104b520 HTTP/1.1" status: 204 len: 173 time: 1.1656029 May 25 16:05:17.292883 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97e3a013-bd7d-4d14-bf0c-4565027a5505 req-45a3569d-809d-4ca8-8662-ba716068f05b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:17.294351 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97e3a013-bd7d-4d14-bf0c-4565027a5505 req-45a3569d-809d-4ca8-8662-ba716068f05b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0899227 May 25 16:05:17.351433 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97e3a013-bd7d-4d14-bf0c-4565027a5505 req-acdfbdee-32ef-4b36-b5e7-1d24c2d38212 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0501785 May 25 16:05:17.927638 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3689e1ba-d65d-4055-baec-a02d36386115 req-083f8fcb-6899-4407-ab51-dd88938dd4ee tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=a7885d47-fd9e-463a-be96-bdd6c5b6988f HTTP/1.1" status: 200 len: 210 time: 0.0426970 May 25 16:05:18.571222 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-90e848c6-ed32-4544-b4d4-98a4fe539fa7 req-89d25970-8483-4d73-8e9a-b8de0a81052f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:18.575146 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-90e848c6-ed32-4544-b4d4-98a4fe539fa7 req-89d25970-8483-4d73-8e9a-b8de0a81052f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1142004 May 25 16:05:18.650702 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-90e848c6-ed32-4544-b4d4-98a4fe539fa7 req-a89e3b2e-4071-4f54-9828-12ff0b88aece tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0686862 May 25 16:05:19.386857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-9fb9581c-76a0-4165-8557-b03ac9ddc224 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:19.388707 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-9fb9581c-76a0-4165-8557-b03ac9ddc224 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 809 time: 0.1550233 May 25 16:05:19.408262 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-e96a0dd3-c864-4964-8b63-3e19e669a0ad tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/quotas/9e6e907c5841468a85bc59f21b26372d HTTP/1.1" status: 200 len: 375 time: 0.0112286 May 25 16:05:19.458132 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-75d8b20c-c25a-45fc-9e03-d130990b50ab tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0436635 May 25 16:05:19.600251 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-8d661883-be45-45d5-95cf-0c14fac700aa service neutron] 10.222.0.82 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1338277 May 25 16:05:19.605532 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 08ee0981-245d-4233-9301-2d6fa104b520 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:05:19.643007 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 08ee0981-245d-4233-9301-2d6fa104b520 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:05:19.643356 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 08ee0981-245d-4233-9301-2d6fa104b520 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:05:19.744187 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-6d3c8cfa-7f5d-41aa-b1a9-8e9fdf0c5232 service neutron] 10.222.0.82 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1371827 May 25 16:05:20.209617 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b250374e-8700-4d0b-894c-8e5ef1b1b76c req-18d3b36c-cd66-423e-8ac2-1c10307a8bfd tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 210 time: 0.0588837 May 25 16:05:20.245938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bc7a4fd1-6e5b-435d-a72c-144e2843a5b7 req-3df09ce5-c9ab-4bc4-aae0-bd280ac0bc25 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:20.251598 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bc7a4fd1-6e5b-435d-a72c-144e2843a5b7 req-3df09ce5-c9ab-4bc4-aae0-bd280ac0bc25 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2049770 May 25 16:05:20.309017 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bc7a4fd1-6e5b-435d-a72c-144e2843a5b7 req-1c898072-4207-4351-aa82-93d331dc22d6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0548310 May 25 16:05:23.876729 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5f016a2c-8dce-4198-a0c1-43dfe75aac38 req-f51c9b48-82f2-475b-a073-b8a19b2a0e01 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 210 time: 0.0752313 May 25 16:05:23.972106 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6041d9e-4fd1-41c3-ac1c-47da04549a62 req-1809b553-bcd3-4e39-a5e9-b18b7855a181 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:23.973698 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6041d9e-4fd1-41c3-ac1c-47da04549a62 req-1809b553-bcd3-4e39-a5e9-b18b7855a181 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0952463 May 25 16:05:24.021987 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6041d9e-4fd1-41c3-ac1c-47da04549a62 req-1ee12a56-b0d9-4202-91a7-4ea1bed78689 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0414386 May 25 16:05:24.260900 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:05:25.092234 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cda7056f-3c82-4991-aad8-9c25d9e39256 req-befec6a6-5a80-4a92-a338-cca2210f1f23 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 210 time: 0.0439849 May 25 16:05:25.290025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db0fb7fa-35d6-45ce-82f9-3e9016ac3836 req-8c147f04-4354-45d8-bd27-dedf4e150ec6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:25.291558 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db0fb7fa-35d6-45ce-82f9-3e9016ac3836 req-8c147f04-4354-45d8-bd27-dedf4e150ec6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0982544 May 25 16:05:25.341939 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db0fb7fa-35d6-45ce-82f9-3e9016ac3836 req-0d5f8360-e93b-4d31-8c1a-5d813e1d288f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0443799 May 25 16:05:25.600543 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-f04eeca1-61db-483c-b94d-771f82bb3e81 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:25.603526 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-f04eeca1-61db-483c-b94d-771f82bb3e81 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 809 time: 0.1604707 May 25 16:05:25.629959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Request body: {'port': {'device_id': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e', 'admin_state_up': True, 'tenant_id': '9e6e907c5841468a85bc59f21b26372d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:25.775406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:25.780890 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Usage tracker for resource:port and tenant:9e6e907c5841468a85bc59f21b26372d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:25.784475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:25.795444 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:25.962258 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Made reservation on behalf of 9e6e907c5841468a85bc59f21b26372d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:25.966406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:26.168660 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Allocated IP 172.20.0.8 (54cf369d-e27b-4d90-a4f4-d729a1eb711e/4fa131e2-5aa0-4f62-bf74-2a67a79ad81c/11c6b8f1-f51c-4048-aa6e-456bb22bb7c5) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:05:26.295856 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0a660e09-6dd0-4624-abcd-e8ce1b7ec324 req-1cf0e0dc-761c-46ba-a519-b388da0dfb2f tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 210 time: 0.0633807 May 25 16:05:26.505063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:26.660059 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:26.672543 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-20bba629-0ed0-4f9d-9eac-634557334aeb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:26.737733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Network 54cf369d-e27b-4d90-a4f4-d729a1eb711e is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:05:26.740854 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:26.744920 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:05:26.750386 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:26.751024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:26.768512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Pushing event updated for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:26.778271 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:26.790817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-810b5bc4-9e12-44dd-9a5a-4b35e3270a12 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:26.792308 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-810b5bc4-9e12-44dd-9a5a-4b35e3270a12 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.1623855 May 25 16:05:26.807516 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-c381f753-82ec-44f7-bd66-087b9bc69f38 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/quotas/ed476c44700b47648958b0d2e020320d HTTP/1.1" status: 200 len: 375 time: 0.0092993 May 25 16:05:26.978652 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-e89208f7-f43c-4972-b14f-25083b91961c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1656992 May 25 16:05:26.986788 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3607790 May 25 16:05:27.011509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Request body: {'port': {'device_id': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:27.129108 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-1371c656-b4c3-4ff4-9897-393048d157d6 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1431561 May 25 16:05:27.135744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:27.291473 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-2032ac5a-4c31-4ffe-ab8c-2b3fdf2a2156 service neutron] 10.222.0.82 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1555243 May 25 16:05:27.563771 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:05:27.563771 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:27.584285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Transition to ACTIVE for port object 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:05:27.669706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bcd64b57-9df9-4c2f-9763-ca0f1126fb02 req-ee84a321-b2a1-4c52-a909-5aec50221efc tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:27.671509 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bcd64b57-9df9-4c2f-9763-ca0f1126fb02 req-ee84a321-b2a1-4c52-a909-5aec50221efc tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 931 time: 0.1270728 May 25 16:05:27.716677 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:27.727485 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4b67769e-4410-42c9-bce9-61ea0110de5b tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:27.775171 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bcd64b57-9df9-4c2f-9763-ca0f1126fb02 req-0213f7c0-5348-407a-b605-d21a7b38bfec tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0970330 May 25 16:05:27.822283 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Pushing event updated for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:27.837117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:27.904401 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5073caca-6407-408d-8062-c05207ccc255 req-8e12a84e-8969-47be-83c2-7d0ec61db41f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 210 time: 0.0669308 May 25 16:05:28.034700 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86e0674a-dc32-4310-8fac-bb6e344dd9b5 req-37b4eb09-9989-4c8b-b6e1-c5669e43eebf tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:28.036309 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86e0674a-dc32-4310-8fac-bb6e344dd9b5 req-37b4eb09-9989-4c8b-b6e1-c5669e43eebf tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0961752 May 25 16:05:28.092795 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86e0674a-dc32-4310-8fac-bb6e344dd9b5 req-9acc9781-37d8-4133-a050-88e18be8163d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0480840 May 25 16:05:28.095541 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Attempting to bind port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:05:28.095883 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Attempting to bind port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:05:28.096278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Attempting to bind port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:28.102813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:05:28.103176 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Attempting to bind port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 on network 54cf369d-e27b-4d90-a4f4-d729a1eb711e {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:28.109502 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 5, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:05:28.109842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Checking segment: {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:05:28.110397 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Bound using segment: {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:05:28.110946 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Bound port: 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'd91de080-4e40-4f71-aea3-f86a8f72864b', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 918, 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:05:28.111303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:28.240903 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:05:28.246724 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:05:28.385160 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:28.389143 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:28.456012 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:28.456667 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:28.458357 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] 10.222.0.42 "PUT /v2.0/ports/11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 HTTP/1.1" status: 200 len: 1093 time: 1.4516861 May 25 16:05:28.459024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:28.484369 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Pushing event updated for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:28.487725 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:28.727186 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5', 'status': 'completed', 'code': 200} May 25 16:05:28.779881 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-1eae9ee3-440f-49fa-a51c-c54cab017f3f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:28.781371 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-1eae9ee3-440f-49fa-a51c-c54cab017f3f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 1096 time: 0.1005294 May 25 16:05:28.894118 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-7b8483b8-b62f-4576-95d3-4986adee8b27 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 HTTP/1.1" status: 200 len: 217 time: 0.0767372 May 25 16:05:29.022298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1cee002b-183c-4681-8226-2a3618ce7348 req-17cf2e77-0508-4f8e-935e-302e871c3cab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:29.026861 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-17cf2e77-0508-4f8e-935e-302e871c3cab service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c HTTP/1.1" status: 200 len: 849 time: 0.1070783 May 25 16:05:29.096208 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-97bbee90-1330-48bb-941f-955eac041057 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=54cf369d-e27b-4d90-a4f4-d729a1eb711e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0592344 May 25 16:05:29.212364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-973fef92-d6bd-4834-93b3-ff5d9c994bf7 req-997cf376-91e2-4cb5-88be-ee8ceb920b18 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:29.213910 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-973fef92-d6bd-4834-93b3-ff5d9c994bf7 req-997cf376-91e2-4cb5-88be-ee8ceb920b18 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 943 time: 0.0956585 May 25 16:05:29.258614 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-f97db3a2-0bbf-4921-a88c-3a1d9816c4cc service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1542475 May 25 16:05:29.262162 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-973fef92-d6bd-4834-93b3-ff5d9c994bf7 req-aa2bd82a-b84b-4038-a902-3147b9c86cb5 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0418980 May 25 16:05:29.460153 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a4f6d2c9-dd27-4f3b-84d7-96ee529c61b1 req-8d3fb90d-e962-4bb5-8f79-159e546f6ab1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 210 time: 0.0642219 May 25 16:05:29.486902 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1cee002b-183c-4681-8226-2a3618ce7348 req-4dbbd1e2-5111-49c8-9f3e-618ef3e21490 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2201588 May 25 16:05:29.813025 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-fe06c5f4-9669-4610-9c51-fa35bc1c1afe service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:29.814478 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-fe06c5f4-9669-4610-9c51-fa35bc1c1afe service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=9e6e907c5841468a85bc59f21b26372d&device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 1096 time: 0.0750656 May 25 16:05:29.986520 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-bce838d3-74ad-4ad5-b003-a540656fd8ef service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:29.986520 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-bce838d3-74ad-4ad5-b003-a540656fd8ef service neutron] 10.222.0.42 "GET /v2.0/networks?id=54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 200 len: 910 time: 0.1533751 May 25 16:05:30.119013 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-dc63286d-be93-4b30-9b50-a747df33d3a7 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.8&port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 HTTP/1.1" status: 200 len: 217 time: 0.1244524 May 25 16:05:30.188533 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-10570f04-363e-4151-9430-87fcc87dce47 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:30.190002 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-10570f04-363e-4151-9430-87fcc87dce47 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c HTTP/1.1" status: 200 len: 849 time: 0.0600488 May 25 16:05:30.242409 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-4327c2e2-1865-4642-8c58-2fc2a9886079 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=54cf369d-e27b-4d90-a4f4-d729a1eb711e&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0426040 May 25 16:05:30.423995 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-97fb8de8-8a23-4bdf-a064-9a303d09ad49 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1599219 May 25 16:05:30.472003 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a0e7f9f8-8750-4a90-b905-82f45557e180 req-4b1da688-3430-4957-bb0e-a25e8a9587a9 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:30.473449 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a0e7f9f8-8750-4a90-b905-82f45557e180 req-4b1da688-3430-4957-bb0e-a25e8a9587a9 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 943 time: 0.0963137 May 25 16:05:30.525106 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a0e7f9f8-8750-4a90-b905-82f45557e180 req-d507942b-4404-4733-8247-b950fadb618f tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0437756 May 25 16:05:30.589580 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05fd0288-f6c7-4cdc-9bef-7d84faf7150a req-42f37ce1-4aa7-4e61-addc-e57048914ef2 service neutron] 10.222.0.42 "GET /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1577876 May 25 16:05:30.634075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-50987a0b-c408-48d4-b49d-9b70c9391c9f req-a40880ab-9798-4466-bc05-214632ad81d4 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 210 time: 0.0519807 May 25 16:05:31.003268 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-63fcc4ce-68a4-4ded-9438-e9cbc862a6cc tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:31.004910 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-63fcc4ce-68a4-4ded-9438-e9cbc862a6cc tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/networks?id=9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 200 len: 814 time: 0.2412724 May 25 16:05:31.031462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Request body: {'port': {'device_id': '2e9c509c-41ab-401c-a248-d074d815b353', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'admin_state_up': True, 'tenant_id': 'ed476c44700b47648958b0d2e020320d'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:31.182946 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:31.190633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Usage tracker for resource:port and tenant:ed476c44700b47648958b0d2e020320d is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:31.194090 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Quota usage for port was recalculated. Used quota:3. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:31.207532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:31.300275 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Made reservation on behalf of ed476c44700b47648958b0d2e020320d for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:31.312081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:31.501633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Allocated IP 172.20.0.10 (9123c0ac-e64c-41a9-8d68-d98246284959/b19f7657-6038-4479-b9ec-9e4e92c69bb0/4a1a8795-a2e2-4bdb-a937-fc10b821e188) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:05:31.668321 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:05:31.776113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fedeff6f-d381-4ed7-8721-a0b80a35a381 req-f70f091c-d7e8-4bc5-8309-748ad26262dd tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:31.783337 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fedeff6f-d381-4ed7-8721-a0b80a35a381 req-f70f091c-d7e8-4bc5-8309-748ad26262dd tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 943 time: 0.1392736 May 25 16:05:31.823782 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:31.835506 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-33190714-f666-43b9-9251-11edea77b2ca req-87f1d8f8-169c-4fc3-9402-d9279649858e tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 210 time: 0.0492935 May 25 16:05:31.891819 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fedeff6f-d381-4ed7-8721-a0b80a35a381 req-9f24944b-ef68-45f1-9d51-d01ae4b6c084 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1070409 May 25 16:05:31.921824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43890) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:31.936475 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:31.960334 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5', 'network_id': '54cf369d-e27b-4d90-a4f4-d729a1eb711e', 'port_id': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5', 'mac_address': 'fa:16:3e:14:a7:89', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 918, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4fa131e2-5aa0-4f62-bf74-2a67a79ad81c', 'ip_address': '172.20.0.8'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:05:31.985972 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:31.996030 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c0424492-41ba-4248-a9da-0417fc0a3344 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:32.032820 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.052664 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc', '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', '2206a32c-5500-4208-9123-7aab17fc53c7'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:05:32.078079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Network 9123c0ac-e64c-41a9-8d68-d98246284959 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:05:32.081170 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:32.085530 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:05:32.092541 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:32.093019 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.115852 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:05:32.119374 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event updated for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:32.121599 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:32.139998 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:32.166710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.184603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:32.189006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Usage tracker for resource:security_group_rule and tenant:a6e9ae91103b4814b179f5102958b5f4 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:32.192047 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:32.198158 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:32.220220 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.1924677 May 25 16:05:32.245086 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Request body: {'port': {'device_id': '2e9c509c-41ab-401c-a248-d074d815b353', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:32.249675 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.254014 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:32.312361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.358199 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:32.368985 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.380832 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.381183 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.381515 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.381846 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.382202 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.382532 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.382997 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.383327 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.383673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.383994 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.384315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.384629 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:05:32.390098 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:32.406206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.441741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:32.468868 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=a6e9ae91103b4814b179f5102958b5f4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5458236 May 25 16:05:32.475813 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43902) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:32.570401 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.572529 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=933120df-19e1-4418-8ff5-efe83b9862f4,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:32.576419 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.135s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:32.613938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.774384 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 4a1a8795-a2e2-4bdb-a937-fc10b821e188 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:05:32.782126 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.796058 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.798797 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Transition to ACTIVE for port object 4a1a8795-a2e2-4bdb-a937-fc10b821e188 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:05:32.801387 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c06d8990-d013-47e5-bab0-452c70c004c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:32.814450 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=933120df-19e1-4418-8ff5-efe83b9862f4,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:32.824202 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:32.830566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.831984 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/933120df-19e1-4418-8ff5-efe83b9862f4 HTTP/1.1" status: 204 len: 168 time: 0.3588858 May 25 16:05:32.945674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:32.954004 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:32.974695 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43924) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:32.993268 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['11c6b8f1-f51c-4048-aa6e-456bb22bb7c5'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:05:33.128579 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Pushing event updated for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:33.136320 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.182s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:33.154430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Attempting to bind port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:05:33.154839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Attempting to bind port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:05:33.155262 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Attempting to bind port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:33.160377 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 on network 9123c0ac-e64c-41a9-8d68-d98246284959 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:05:33.160795 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Attempting to bind port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 on network 9123c0ac-e64c-41a9-8d68-d98246284959 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:33.169506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 5, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:05:33.169850 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Checking segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:05:33.170241 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Bound using segment: {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:05:33.170764 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Bound port: 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'be9421e6-1214-4783-b6f4-e62225b27ecf', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 948, 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:05:33.171068 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.179143 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Deleting subnet 1c61f546-0daa-4a78-b2f0-d09f03693bd8 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:05:33.183563 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.249331 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.294189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:05:33.298556 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:05:33.316547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f37b06d3-9c34-4c96-975d-2fe14d6e3216 req-fa0f8fdf-7763-4202-b6e6-5fb2503b3195 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:33.318116 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f37b06d3-9c34-4c96-975d-2fe14d6e3216 req-fa0f8fdf-7763-4202-b6e6-5fb2503b3195 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 944 time: 0.3072088 May 25 16:05:33.340596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-440aca1f-5876-4708-ab27-79d7329503b9 req-11643fb3-11b5-4fc9-885c-2a737eef53cf tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:33.342149 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-440aca1f-5876-4708-ab27-79d7329503b9 req-11643fb3-11b5-4fc9-885c-2a737eef53cf tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 944 time: 0.2643571 May 25 16:05:33.381510 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.414863 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.423424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f78fcc2-717f-40de-8cbb-a5023907d7a1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:33.455944 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f37b06d3-9c34-4c96-975d-2fe14d6e3216 req-6384f10d-6a09-4695-b7fb-19d707510d35 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1289296 May 25 16:05:33.498410 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-440aca1f-5876-4708-ab27-79d7329503b9 req-36341ea2-c674-4d7b-adca-1f2f99ef4d05 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1374779 May 25 16:05:33.501495 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:33.501859 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.503587 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] 10.222.0.42 "PUT /v2.0/ports/4a1a8795-a2e2-4bdb-a937-fc10b821e188 HTTP/1.1" status: 200 len: 1094 time: 1.2630606 May 25 16:05:33.504121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:33.532316 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Pushing event updated for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:33.536229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:33.598845 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188', 'status': 'completed', 'code': 200} May 25 16:05:33.774893 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-362fb528-dac6-45a9-a7fc-e106fe48a7c0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:33.776953 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-362fb528-dac6-45a9-a7fc-e106fe48a7c0 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 1097 time: 0.0966122 May 25 16:05:33.927873 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.934305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f21ec36d-9fb5-47da-bf71-0e74f02a7da3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:33.970743 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1c61f546-0daa-4a78-b2f0-d09f03693bd8,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:33.975889 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:33.980192 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:33.987849 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/1c61f546-0daa-4a78-b2f0-d09f03693bd8 HTTP/1.1" status: 204 len: 168 time: 1.0108635 May 25 16:05:33.990829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43960) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:34.056811 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-ee443724-9c19-4bc0-b06d-660ea8aa40a4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.058323 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-ee443724-9c19-4bc0-b06d-660ea8aa40a4 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=ed476c44700b47648958b0d2e020320d&shared=False HTTP/1.1" status: 200 len: 915 time: 0.2622671 May 25 16:05:34.108947 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.212540 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-17c3257a-f622-40f1-81f6-79a540209da4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.214001 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2f0ef84f-6d46-4e56-968e-9c6b10bb7c8a req-17c3257a-f622-40f1-81f6-79a540209da4 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1424243 May 25 16:05:34.245413 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.259601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['db850d4d-e971-4d97-b059-7087190eebdd']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:34.272230 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'bbabde65-843a-4b69-8726-c5a9d990a020') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:34.272526 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'bbabde65-843a-4b69-8726-c5a9d990a020') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:34.284162 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.301463 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.309583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Releasing vlan 925 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:05:34.309988 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:05:34.324992 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:05:34.396975 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-4018e9e3-2521-4fbb-89fc-039c46a5446d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.398694 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-4018e9e3-2521-4fbb-89fc-039c46a5446d service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=ed476c44700b47648958b0d2e020320d&device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 1097 time: 0.0842972 May 25 16:05:34.487029 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-990bc4d3-e380-4537-afe0-a00c139adc03 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.10&port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188 HTTP/1.1" status: 200 len: 217 time: 0.0706043 May 25 16:05:34.498808 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.504471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:34.519392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.541646 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] 10.222.0.82 "DELETE /v2.0/networks/db850d4d-e971-4d97-b059-7087190eebdd HTTP/1.1" status: 204 len: 168 time: 0.5491045 May 25 16:05:34.544556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Pushing event deleted for resources: {'Network': ['ID=db850d4d-e971-4d97-b059-7087190eebdd,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:34.549475 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43976) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:34.556849 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:34.592302 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-f3ebab76-590e-4e8c-b57b-c2af552a33d4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.596515 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-f3ebab76-590e-4e8c-b57b-c2af552a33d4 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 200 len: 854 time: 0.0993612 May 25 16:05:34.651941 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-a1fbe6c8-b563-4c64-9986-38430023ffd7 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=9123c0ac-e64c-41a9-8d68-d98246284959&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0482392 May 25 16:05:34.736651 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Deleting subnet 02869cfb-fcb1-40ce-b14b-a2b3200f595b {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:05:34.742000 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.813480 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-885739ce-1ed0-4c8d-b8a2-b23d5b226f29 service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1529899 May 25 16:05:34.814946 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.878591 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18163cf0-450d-4c28-a6dd-a24f71d3c403 req-357d4a3e-1bdb-4b0d-979a-5ef5f29f2462 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.880250 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-18163cf0-450d-4c28-a6dd-a24f71d3c403 req-357d4a3e-1bdb-4b0d-979a-5ef5f29f2462 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 944 time: 0.2780058 May 25 16:05:34.891259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c4fe0043-a89c-4c9f-a39c-fd809254f526 req-3ed658c5-84ce-4ec8-9dac-0af7bc270c88 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:34.895455 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c4fe0043-a89c-4c9f-a39c-fd809254f526 req-3ed658c5-84ce-4ec8-9dac-0af7bc270c88 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 944 time: 0.2423511 May 25 16:05:34.948318 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:34.962734 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3ca33f99-2d8a-44de-947b-40a292f02361 req-29e5f314-4ed5-4194-88f7-643bac16c23d service neutron] 10.222.0.42 "GET /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1323664 May 25 16:05:35.021959 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-18163cf0-450d-4c28-a6dd-a24f71d3c403 req-07f1c3b8-5482-4c77-9eeb-93e58eaf4f94 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1231587 May 25 16:05:35.026676 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c4fe0043-a89c-4c9f-a39c-fd809254f526 req-bc18db12-6479-42cd-997a-018c3afd6fa8 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1247561 May 25 16:05:35.396711 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.402647 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7c2e8a63-3d15-443a-b08e-7276d0b33adc admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:35.439415 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Pushing event deleted for resources: {'Subnet': ['ID=02869cfb-fcb1-40ce-b14b-a2b3200f595b,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:35.444087 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:35.448496 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.455283 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] 10.222.0.82 "DELETE /v2.0/subnets/02869cfb-fcb1-40ce-b14b-a2b3200f595b HTTP/1.1" status: 204 len: 168 time: 0.9047408 May 25 16:05:35.457646 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 43994) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:35.574335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.648575 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f2779e99-2475-480f-8621-77a05395e85e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.701128 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:05:35.783314 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.798210 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['cb9233b3-2734-4a6a-9749-ee4b98a4a54a']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:35.811583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '55bc8fb5-7948-478b-9d7a-e4b5d9e1ac5d') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:35.811877 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '55bc8fb5-7948-478b-9d7a-e4b5d9e1ac5d') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:05:35.826073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.855664 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.862566 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Releasing vlan 905 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:05:35.862849 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.864873 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:05:35.870219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:05:35.870650 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:05:35.877009 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:05:36.020745 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:36.065280 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:36.071974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:36.076219 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:36.077105 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:36.091288 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:36.109481 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Pushing event deleted for resources: {'Network': ['ID=cb9233b3-2734-4a6a-9749-ee4b98a4a54a,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:36.124500 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] 10.222.0.82 "DELETE /v2.0/networks/cb9233b3-2734-4a6a-9749-ee4b98a4a54a HTTP/1.1" status: 204 len: 168 time: 0.6649058 May 25 16:05:36.126551 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ffdabbca-81c2-487f-9bb2-c43d2d1bb9e2 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:36.174027 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:36.176981 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:36.326422 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:36.330051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:36.418729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1b2ea561-1798-4339-b924-13721b2b17e4 req-479f255b-b105-4124-9629-4169c9ea228d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:36.420770 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1b2ea561-1798-4339-b924-13721b2b17e4 req-479f255b-b105-4124-9629-4169c9ea228d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1912291 May 25 16:05:36.435427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31c21f58-d0d9-4a19-8421-551972df3a9c req-7576bbd5-2035-47f6-94a5-06b8e01fe85c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:36.436857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31c21f58-d0d9-4a19-8421-551972df3a9c req-7576bbd5-2035-47f6-94a5-06b8e01fe85c tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 944 time: 0.1318090 May 25 16:05:36.512887 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1b2ea561-1798-4339-b924-13721b2b17e4 req-61f7655e-c22a-4cea-a84e-e944cf2dd2cd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0862155 May 25 16:05:36.515555 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31c21f58-d0d9-4a19-8421-551972df3a9c req-67bc91a3-45fa-4f17-b58e-02952259ce72 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0649824 May 25 16:05:36.580724 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5', 'code': 200} May 25 16:05:36.726150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9eeb622d-ad2d-4de1-8bff-4bc81777188a req-81848c16-10a1-4577-b479-bf8c5bad740e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:36.728198 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9eeb622d-ad2d-4de1-8bff-4bc81777188a req-81848c16-10a1-4577-b479-bf8c5bad740e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.0998132 May 25 16:05:36.780991 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9eeb622d-ad2d-4de1-8bff-4bc81777188a req-c544143e-be27-4f4a-a7b7-862fd126c028 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0451696 May 25 16:05:36.801747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44032) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:36.857795 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9a4839c3-93b7-4f46-8e6a-1b0e67dd69ad admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=6055d0e4eebd4015979060d95f3f0f8c&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0526147 May 25 16:05:36.858192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44036) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:36.921893 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:36.956774 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:37.066746 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b550b0e0-ad1c-43fe-8a61-fae078c0d4cc req-c2b29259-0c54-4064-b77c-6c3fd2303412 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:37.068240 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b550b0e0-ad1c-43fe-8a61-fae078c0d4cc req-c2b29259-0c54-4064-b77c-6c3fd2303412 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 1098 time: 0.0779061 May 25 16:05:37.112038 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b550b0e0-ad1c-43fe-8a61-fae078c0d4cc req-64676495-4fd1-40fa-9bd6-cfa668167c1d service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0351024 May 25 16:05:37.176140 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:37.181524 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-506f93df-e9e9-45ee-be53-4cece6f13104 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:37.199117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d60c2e4c-2be2-42d4-99d5-8e3616307a3b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:37.201650 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:37.214841 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:37.215997 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/d60c2e4c-2be2-42d4-99d5-8e3616307a3b HTTP/1.1" status: 204 len: 168 time: 0.3568156 May 25 16:05:37.723518 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4a1a8795-a2e2-4bdb-a937-fc10b821e188 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:05:37.724353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c77eb9f-3c62-48f8-8d98-8e007c7f419e req-01cc5358-aac7-4cff-9375-068fa17a4332 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:37.726207 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9c77eb9f-3c62-48f8-8d98-8e007c7f419e req-01cc5358-aac7-4cff-9375-068fa17a4332 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.0915153 May 25 16:05:37.781223 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9c77eb9f-3c62-48f8-8d98-8e007c7f419e req-d454377d-acd6-42c9-9382-b75433a106ad tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0481923 May 25 16:05:37.877292 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:37.892653 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '4a1a8795-a2e2-4bdb-a937-fc10b821e188', 'network_id': '9123c0ac-e64c-41a9-8d68-d98246284959', 'port_id': '4a1a8795-a2e2-4bdb-a937-fc10b821e188', 'mac_address': 'fa:16:3e:4b:fa:fb', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 948, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'b19f7657-6038-4479-b9ec-9e4e92c69bb0', 'ip_address': '172.20.0.10'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:05:37.959176 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:37.976360 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86e1c250-1ad8-444d-bc96-140c82ea517d req-6ce4247d-5693-4f28-9a12-3e397fdbdfd8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:37.977390 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 4a1a8795-a2e2-4bdb-a937-fc10b821e188 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:05:37.977815 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86e1c250-1ad8-444d-bc96-140c82ea517d req-6ce4247d-5693-4f28-9a12-3e397fdbdfd8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 944 time: 0.0903592 May 25 16:05:37.995168 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:38.024954 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86e1c250-1ad8-444d-bc96-140c82ea517d req-cd759368-4ca3-4499-8c33-9f556f9c7b35 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0399511 May 25 16:05:38.170345 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.173213 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:38.259166 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:38.262822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44074) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:38.265596 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.267516 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:38.287869 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3fcf3a81-6589-47ce-a788-6f98d2a6eff3 req-d9d59a95-3e2a-4b04-ad20-4502ba88c153 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:38.290594 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3fcf3a81-6589-47ce-a788-6f98d2a6eff3 req-d9d59a95-3e2a-4b04-ad20-4502ba88c153 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.1367803 May 25 16:05:38.327461 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-08a78b11-d55d-4a11-a339-a17b6fed0e76 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=0f31b747aa294350b3392970ce891fe3&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0633423 May 25 16:05:38.331126 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44078) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:38.362510 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3fcf3a81-6589-47ce-a788-6f98d2a6eff3 req-8f257475-082c-43e5-aab4-6d116a0ec336 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0660183 May 25 16:05:38.419387 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.452991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.499100 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-472ede0f-6340-47d4-822b-33d4ff7d834f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.584585 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.595231 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:38.607000 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a0327111-6609-4f1e-a645-1f604163df8b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:38.609663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:38.616040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:38.633341 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-0d610024-b37b-4084-839f-66bbba6f7a24 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/a0327111-6609-4f1e-a645-1f604163df8b HTTP/1.1" status: 204 len: 168 time: 0.2882233 May 25 16:05:38.749022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['4a1a8795-a2e2-4bdb-a937-fc10b821e188'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:05:38.801530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44090) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:39.124393 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4fb0ac81-1b50-4df8-af26-c8b54aad698e req-43f202ef-0f29-4f35-b77c-b73a9c15ee71 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:39.125927 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4fb0ac81-1b50-4df8-af26-c8b54aad698e req-43f202ef-0f29-4f35-b77c-b73a9c15ee71 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1285427 May 25 16:05:39.194154 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4fb0ac81-1b50-4df8-af26-c8b54aad698e req-6382c976-4db4-4ad3-8376-4fca1762546d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0616534 May 25 16:05:39.196842 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:39.326636 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:39.352549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:39.357849 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Usage tracker for resource:security_group_rule and tenant:bc28e5c5870f4ae88be76f850903f270 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:39.361942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:39.370839 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:39.409612 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cde57d1e-b3a2-4e36-bb00-e893c338be41 req-d9556a58-b76f-442e-b589-150a98e8692f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:39.411188 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cde57d1e-b3a2-4e36-bb00-e893c338be41 req-d9556a58-b76f-442e-b589-150a98e8692f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 945 time: 0.1867192 May 25 16:05:39.478794 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cde57d1e-b3a2-4e36-bb00-e893c338be41 req-fb2bfbf8-db04-4ecd-80fc-1e9d7d84f392 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0597541 May 25 16:05:39.572664 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:39.610517 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a31e3a81-c67f-4b32-9b59-bb6f0cfd39b9 req-30e82758-f5c6-4ab3-862b-31499c5c141a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:39.612081 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a31e3a81-c67f-4b32-9b59-bb6f0cfd39b9 req-30e82758-f5c6-4ab3-862b-31499c5c141a tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.1118264 May 25 16:05:39.664121 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a31e3a81-c67f-4b32-9b59-bb6f0cfd39b9 req-293e7f3a-6256-479c-bbd6-495fb63763e9 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0452838 May 25 16:05:39.756399 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:39.771193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d3774bc1-5ba7-4b3d-bb5f-ca17b331d0c5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:39.810658 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=bc28e5c5870f4ae88be76f850903f270&name=default HTTP/1.1" status: 200 len: 2979 time: 1.0069182 May 25 16:05:39.816059 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a74ccf8e-ac14-4e14-ba83-63fdf57de73c,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:39.819350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44124) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:39.821688 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.055s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:39.887292 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:39.918086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.071287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.075159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c5837d15-3c24-4654-a195-3f52172b0eb2 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:40.090942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a74ccf8e-ac14-4e14-ba83-63fdf57de73c,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:40.093103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:40.095748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.097110 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/a74ccf8e-ac14-4e14-ba83-63fdf57de73c HTTP/1.1" status: 204 len: 168 time: 0.2766399 May 25 16:05:40.189943 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44136) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:40.378430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Request body: {'network': {'name': 'tempest-VolumesSnapshotTestJSON-999624056-network', 'tenant_id': 'e8bb2885c2ca420b852158159d30e3b4'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:40.388312 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:40.393189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Usage tracker for resource:network and tenant:e8bb2885c2ca420b852158159d30e3b4 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:40.396293 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:40.410291 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:40.438572 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.446758 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8b383016-dac2-4ddf-bce3-26a271962f31 req-665801a7-65c3-4e2a-adf8-d459403ff61f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:40.448328 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8b383016-dac2-4ddf-bce3-26a271962f31 req-665801a7-65c3-4e2a-adf8-d459403ff61f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1322613 May 25 16:05:40.520388 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8b383016-dac2-4ddf-bce3-26a271962f31 req-7bdaa9a1-0167-431d-9dba-66d19a3afb77 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0652082 May 25 16:05:40.533780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Made reservation on behalf of e8bb2885c2ca420b852158159d30e3b4 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:40.536431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.554208 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.573439 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:40.578666 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Usage tracker for resource:security_group_rule and tenant:e8bb2885c2ca420b852158159d30e3b4 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:40.582081 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:40.589471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:40.782162 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.797121 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a8157968-236a-4cad-8ec4-7bf1c0de29b1 req-3c4f3143-861d-41e5-b4f7-df0408e11f3f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:40.805358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8157968-236a-4cad-8ec4-7bf1c0de29b1 req-3c4f3143-861d-41e5-b4f7-df0408e11f3f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 945 time: 0.1711674 May 25 16:05:40.858008 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.862913 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-38a8228a-f353-4535-95c8-ad13bb6483d3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:40.872982 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a8157968-236a-4cad-8ec4-7bf1c0de29b1 req-c3cb5b34-3f39-48e4-a373-9f598ee3a151 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0668564 May 25 16:05:40.893323 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] vlan segment allocate from pool success with {'vlan_id': 947, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:05:40.906479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bb0c097f-e86f-43cf-a661-bffb61d1fb33,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:40.910789 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:40.911601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:40.911937 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Added segment f1cda128-299e-4975-835f-ff55ee624d0a of type vlan for network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f May 25 16:05:40.940167 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:41.133179 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:41.133468 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:41.134602 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:41.135211 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:41.138747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:41.312335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Pushing event updated for resources: {'Network': ['ID=610dbdeb-d88d-45f7-b872-9563bbbe7c3f,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:41.315146 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:41.323459 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:05:41.402992 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 886 time: 1.2118866 May 25 16:05:41.405944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44182) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:41.416791 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Request body: {'subnet': {'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f', 'cidr': '172.20.0.0/28', 'name': 'tempest-VolumesSnapshotTestJSON-999624056-subnet', 'tenant_id': 'e8bb2885c2ca420b852158159d30e3b4', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:41.425350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:41.432662 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Usage tracker for resource:subnet and tenant:e8bb2885c2ca420b852158159d30e3b4 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:41.437934 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:41.448774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:41.573210 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Made reservation on behalf of e8bb2885c2ca420b852158159d30e3b4 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:41.590798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:41.729132 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4a1a8795-a2e2-4bdb-a937-fc10b821e188 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:05:41.976142 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55c24007-d401-46de-8fa0-3c105e58b040 req-ca77551c-a79c-437c-b982-f7371597c00d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:41.977712 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55c24007-d401-46de-8fa0-3c105e58b040 req-ca77551c-a79c-437c-b982-f7371597c00d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2123702 May 25 16:05:41.990493 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:05:41.998645 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:05:41.998645 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:42.029083 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-55c24007-d401-46de-8fa0-3c105e58b040 req-d302927f-b7e3-452e-86be-558930095f39 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0452359 May 25 16:05:42.228251 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:42.281449 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:42.293225 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:42.380581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2987e58c-7670-4c39-b220-65c5fe5c6d14 req-5c1e3ede-7d0b-4202-a1ac-e5625c028524 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:42.382668 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2987e58c-7670-4c39-b220-65c5fe5c6d14 req-5c1e3ede-7d0b-4202-a1ac-e5625c028524 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 945 time: 0.1928971 May 25 16:05:42.385167 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:42.387954 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:42.409293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-7b4c1436-0d87-4add-9a72-a83b4fed996b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:42.411548 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-7b4c1436-0d87-4add-9a72-a83b4fed996b service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.0879579 May 25 16:05:42.459498 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2987e58c-7670-4c39-b220-65c5fe5c6d14 req-eeddd9fa-684a-4b2b-b399-e89a181b2ac3 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0678844 May 25 16:05:42.486217 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:42.505009 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:42.534614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31eedc05-11ba-4ebe-b542-cfa459dcfa08 req-02881a7f-4e94-4170-b0a8-1cf7c5bbb474 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:42.536205 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31eedc05-11ba-4ebe-b542-cfa459dcfa08 req-02881a7f-4e94-4170-b0a8-1cf7c5bbb474 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2211349 May 25 16:05:42.537228 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:42.541887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:42.587959 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188', 'code': 200} May 25 16:05:42.673578 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-208585cd-e99d-4041-9e56-c5e118328850 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:42.677213 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-208585cd-e99d-4041-9e56-c5e118328850 service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.2582612 May 25 16:05:42.726430 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31eedc05-11ba-4ebe-b542-cfa459dcfa08 req-8173b725-f77f-43be-8a1f-b0437543ac2a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1503582 May 25 16:05:42.727092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Pushing event updated for resources: {'Subnet': ['ID=2521d219-c255-41fe-80ad-cbccc4e9c1fe,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:42.730693 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:42.755679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Checking segment: {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:05:42.755679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Network hostable DHCP agents. Network: 610dbdeb-d88d-45f7-b872-9563bbbe7c3f, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:05:42.789693 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-b084935b-82e6-4c35-b926-7565e1190fc4 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.1016059 May 25 16:05:42.874181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-ce2278b8-5afb-4382-bc6c-3deccb3ae53c service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:42.876243 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:05:42.885985 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-ce2278b8-5afb-4382-bc6c-3deccb3ae53c service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.0828404 May 25 16:05:42.937934 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-b6317b89-aeb9-4421-b4f2-578cbdeda33a service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0416584 May 25 16:05:43.009148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 610dbdeb-d88d-45f7-b872-9563bbbe7c3f) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:43.021546 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-02dcdca6-daf6-4a11-8cd7-4a4cab62ee57 None None] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:05:43.042603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:43.046733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:43.247767 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d49a2ff-c4d6-4ddb-8d3f-032accaa1e80 req-4601c25b-2207-4cca-bd07-b8f64ec6be3e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:43.249291 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2d49a2ff-c4d6-4ddb-8d3f-032accaa1e80 req-4601c25b-2207-4cca-bd07-b8f64ec6be3e service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 1099 time: 0.1826389 May 25 16:05:43.316869 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-685d42d9-845f-4761-95ac-0c876f0175cd service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3719239 May 25 16:05:43.318567 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b575d307-72f6-4249-b063-cd7b8e145e21 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 1.9110768 May 25 16:05:43.388734 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2d49a2ff-c4d6-4ddb-8d3f-032accaa1e80 req-6d14745b-2eba-4b9e-a380-9aad0a9e12df service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0684175 May 25 16:05:43.423881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6f0ed54-0d10-4580-b7d4-445b40be98dc req-6dc8393e-3154-42d4-afe6-a96886c4dfed tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:43.425521 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f6f0ed54-0d10-4580-b7d4-445b40be98dc req-6dc8393e-3154-42d4-afe6-a96886c4dfed tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1805484 May 25 16:05:43.480102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-283d0949-f6b5-4a7f-a764-59fa6f787765 None None] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:05:43.584208 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f6f0ed54-0d10-4580-b7d4-445b40be98dc req-7ccfec1e-9110-4f97-bcad-f49663166319 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0935059 May 25 16:05:43.661378 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1162ba0-c897-4729-ba97-515353538d1e req-6d1ff567-cf8d-44be-9fb5-d0abdc12618e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:43.661378 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1162ba0-c897-4729-ba97-515353538d1e req-6d1ff567-cf8d-44be-9fb5-d0abdc12618e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.2301919 May 25 16:05:43.671276 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-7d9e791a-ca21-4d1d-9cd6-85b703d74cef req-5e8f03dc-feac-47ef-b2bc-d54347b66397 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3403032 May 25 16:05:43.758100 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1162ba0-c897-4729-ba97-515353538d1e req-6df5ee17-8613-46da-a290-b6c2a4a84bc1 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0941005 May 25 16:05:43.986215 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-648af4b1-e13b-4510-b186-17dc8ebca2a3 req-e19c3ee2-a658-4711-99a1-a4c0c1904cb8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:43.998979 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-648af4b1-e13b-4510-b186-17dc8ebca2a3 req-e19c3ee2-a658-4711-99a1-a4c0c1904cb8 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.2119305 May 25 16:05:44.096698 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-648af4b1-e13b-4510-b186-17dc8ebca2a3 req-8abaee35-1e91-44a8-b31a-b715c0a37a37 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0967395 May 25 16:05:44.127525 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8721bed1-69dc-4c68-82ea-2e050a02f5d0 req-7a355ba9-d92f-49c0-91df-6c215a728afa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:44.132401 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8721bed1-69dc-4c68-82ea-2e050a02f5d0 req-7a355ba9-d92f-49c0-91df-6c215a728afa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1279957 May 25 16:05:44.181285 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8721bed1-69dc-4c68-82ea-2e050a02f5d0 req-e2e7dae8-afa9-4aa2-97b1-88f4f4474df7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0462005 May 25 16:05:44.554691 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a699d091-6df6-4c9a-b9de-e9f1743c710e tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/networks?tenant_id=625c6d6c1b4949ce817bdff51006604f&shared=False HTTP/1.1" status: 200 len: 213 time: 0.0274584 May 25 16:05:44.714733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-8a395b1b-d5f1-46af-94ac-a11a9300edec tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:44.722708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-8a395b1b-d5f1-46af-94ac-a11a9300edec tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.1583283 May 25 16:05:44.773601 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-c7066633-6524-452e-b6fe-6e9bf9359da2 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/quotas/625c6d6c1b4949ce817bdff51006604f HTTP/1.1" status: 200 len: 375 time: 0.0497978 May 25 16:05:44.926186 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-72c4935d-4f7c-4276-b17f-1f5362098e14 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=625c6d6c1b4949ce817bdff51006604f&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1396663 May 25 16:05:44.953351 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-022ef9da-9514-45ca-b31b-4aad8ba149bd req-f5bf1509-4234-4e8e-b153-479481a0e28e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:44.953649 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-022ef9da-9514-45ca-b31b-4aad8ba149bd req-f5bf1509-4234-4e8e-b153-479481a0e28e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2281432 May 25 16:05:45.035560 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-022ef9da-9514-45ca-b31b-4aad8ba149bd req-6bec1c58-49b3-456c-83f6-3a9820b259d8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0716875 May 25 16:05:45.280349 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75b9513b-079e-4020-b15c-ac0f32cf7f44 req-c65d04ec-bb74-4509-9b95-15cf94aed6db tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:45.281874 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75b9513b-079e-4020-b15c-ac0f32cf7f44 req-c65d04ec-bb74-4509-9b95-15cf94aed6db tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.1294577 May 25 16:05:45.345436 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75b9513b-079e-4020-b15c-ac0f32cf7f44 req-1b68743a-cc4d-455b-8c8f-8b277554efd0 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0500512 May 25 16:05:45.361436 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-391cc1df-4e4e-4fc5-9608-670d73729176 req-bd4f0362-1bc1-4e22-9835-4b1b0d88bbef tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:45.363026 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-391cc1df-4e4e-4fc5-9608-670d73729176 req-bd4f0362-1bc1-4e22-9835-4b1b0d88bbef tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.1274395 May 25 16:05:45.449591 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-571cb939-a7a8-4aea-b6e5-db2bb47176a6 req-9bb3c4b2-4010-466f-8e61-f83937888034 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.0594118 May 25 16:05:45.456961 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-391cc1df-4e4e-4fc5-9608-670d73729176 req-c8004d6e-63cf-4b0e-a95f-e311c27cdd19 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0871809 May 25 16:05:45.627557 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b91a25f5-cf11-4275-9280-8a5aedf0cec3 req-6eff9acb-c2d1-4365-9d47-712eff115378 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:45.629144 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b91a25f5-cf11-4275-9280-8a5aedf0cec3 req-6eff9acb-c2d1-4365-9d47-712eff115378 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0819461 May 25 16:05:45.675752 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b91a25f5-cf11-4275-9280-8a5aedf0cec3 req-bbbafd2e-80eb-4395-bc2e-05c6973cba5b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0400531 May 25 16:05:46.283437 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b38600a-49b7-4dbe-bedc-39a6f2be2850 req-bbc15826-3ac3-4cc7-88c0-e6a120b065ba tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:46.284915 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7b38600a-49b7-4dbe-bedc-39a6f2be2850 req-bbc15826-3ac3-4cc7-88c0-e6a120b065ba tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1062438 May 25 16:05:46.336610 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7b38600a-49b7-4dbe-bedc-39a6f2be2850 req-3861efa3-027f-4047-bb67-9011c4036aec tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0452590 May 25 16:05:46.549772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0de8004a-1238-4f60-8c1e-311968f57249 req-d3013782-47b2-4f39-89d0-514d783ef22e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:46.551287 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0de8004a-1238-4f60-8c1e-311968f57249 req-d3013782-47b2-4f39-89d0-514d783ef22e tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.0879567 May 25 16:05:46.644110 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d18a8202-245a-4d65-ba68-7a25bd59ffbc req-5a0d0ec7-012e-434c-b5e3-c6eddcc82697 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.0659633 May 25 16:05:46.653075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0de8004a-1238-4f60-8c1e-311968f57249 req-4ddcbc4d-8336-4900-88a9-0a3d72d67d66 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0952539 May 25 16:05:46.902691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-50bc7a4e-39c5-4a7d-a7d9-4216ed98a0a4 req-57287b3d-481c-4d25-9a0a-66e591b27c46 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:46.904273 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-50bc7a4e-39c5-4a7d-a7d9-4216ed98a0a4 req-57287b3d-481c-4d25-9a0a-66e591b27c46 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.1498120 May 25 16:05:46.937183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0898c6ac-9742-4061-a838-7b34f96dbc18 req-286d91d7-481b-4b9d-8c4b-b7255c19fa82 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:46.938665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0898c6ac-9742-4061-a838-7b34f96dbc18 req-286d91d7-481b-4b9d-8c4b-b7255c19fa82 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1366212 May 25 16:05:46.984778 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-50bc7a4e-39c5-4a7d-a7d9-4216ed98a0a4 req-38d38b16-c3a2-4026-904f-9df586d9953d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0735557 May 25 16:05:47.014009 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0898c6ac-9742-4061-a838-7b34f96dbc18 req-35be404c-a2e1-48df-a4dc-0bdb13bc6ca3 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0685763 May 25 16:05:47.570446 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5b11e0cd-dc45-422c-8f49-6df7046edb6e req-509b22d2-f24a-400b-a0af-2ffbd0a1945c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:47.573027 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5b11e0cd-dc45-422c-8f49-6df7046edb6e req-509b22d2-f24a-400b-a0af-2ffbd0a1945c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1092830 May 25 16:05:47.629651 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5b11e0cd-dc45-422c-8f49-6df7046edb6e req-7ff4ed04-34b5-496b-a006-772d05d2fed6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0509074 May 25 16:05:47.839065 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e69804cd-efb8-492e-bba0-30bcbf501e0d req-425ac859-ec1d-4a69-ad28-32ea29d1373c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.0826790 May 25 16:05:47.883874 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28e1c2d7-eb14-446b-a0b7-7a42204dd36c req-637bd4e2-042a-42d1-ad63-71525963c320 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:47.885358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-28e1c2d7-eb14-446b-a0b7-7a42204dd36c req-637bd4e2-042a-42d1-ad63-71525963c320 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.0976436 May 25 16:05:47.934138 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-28e1c2d7-eb14-446b-a0b7-7a42204dd36c req-c32af6fa-9a8c-4b18-9fa8-5a80f4a35b57 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0420079 May 25 16:05:48.256491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3856a30e-2cbe-49a5-a21c-b4d5ba191489 req-5c4db2ef-dd61-41b4-9c74-98b8e512d6ed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:48.258003 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3856a30e-2cbe-49a5-a21c-b4d5ba191489 req-5c4db2ef-dd61-41b4-9c74-98b8e512d6ed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.1486962 May 25 16:05:48.267097 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5d1c761-48de-476d-b448-5b049dbfe060 req-631bcad8-0093-4b8e-9098-3a87cd005922 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:48.268739 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5d1c761-48de-476d-b448-5b049dbfe060 req-631bcad8-0093-4b8e-9098-3a87cd005922 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1283708 May 25 16:05:48.338392 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3856a30e-2cbe-49a5-a21c-b4d5ba191489 req-945c8ce7-5684-4a83-afb6-208460250fa6 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0688326 May 25 16:05:48.350440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5d1c761-48de-476d-b448-5b049dbfe060 req-94da00aa-f93f-456a-9ee2-0f81fa93e140 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0612371 May 25 16:05:48.778823 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-51874e4d-f250-42af-b21e-66eb82802c55 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:48.779597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-51874e4d-f250-42af-b21e-66eb82802c55 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Synchronizing usage tracker for tenant:9e6e907c5841468a85bc59f21b26372d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:05:48.882519 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-51874e4d-f250-42af-b21e-66eb82802c55 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Unset dirty status for tenant:9e6e907c5841468a85bc59f21b26372d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:05:48.884353 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-51874e4d-f250-42af-b21e-66eb82802c55 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.2102911 May 25 16:05:48.926565 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0d35840-75f6-42a9-a766-aeb99aa97097 req-d1df05ad-4ca6-48d9-acc4-1b210a4ea244 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:48.929634 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0d35840-75f6-42a9-a766-aeb99aa97097 req-d1df05ad-4ca6-48d9-acc4-1b210a4ea244 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1088581 May 25 16:05:49.039152 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0d35840-75f6-42a9-a766-aeb99aa97097 req-a7bcd3e5-9eac-4111-b9c7-e0a8369a1000 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1018703 May 25 16:05:49.044526 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b8f8c4-477b-4477-840f-5b634f8831fa req-4dcaaeab-03c4-4775-95d9-393bfc64cd83 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.0579021 May 25 16:05:49.066919 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Deleting port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:05:49.067261 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:49.245774 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] For port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,segment=NetworkSegment(d91de080-4e40-4f71-aea3-f86a8f72864b),segment_id=d91de080-4e40-4f71-aea3-f86a8f72864b)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:49.246189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:49.250472 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8a02dc05-10fb-4281-a607-f7c8bba45d7b req-693b1b2f-4f1c-4d09-8835-685a179b6b84 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:49.257791 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8a02dc05-10fb-4281-a607-f7c8bba45d7b req-693b1b2f-4f1c-4d09-8835-685a179b6b84 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/ports?device_id=dce17d11-d553-46f1-9694-cc84fef1b4be HTTP/1.1" status: 200 len: 945 time: 0.0987611 May 25 16:05:49.320872 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8a02dc05-10fb-4281-a607-f7c8bba45d7b req-117cf3d0-8e25-4905-866b-0804644b38ac tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.82 "GET /v2.0/security-groups?id=7547d244-8c01-4ddb-8286-ed592f9280b0&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0620978 May 25 16:05:49.387319 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Calling delete_port for 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:05:49.567269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:49.574359 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5d26d736-0bc1-4cf4-9628-c1feef93c942 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:49.606580 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['11c6b8f1-f51c-4048-aa6e-456bb22bb7c5'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:05:49.627351 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Pushing event deleted for resources: {'Port': ['ID=11c6b8f1-f51c-4048-aa6e-456bb22bb7c5,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:49.637410 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:49.723923 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] No DVR routers for this DVR port 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:05:49.728018 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:49.730765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'name': 'network-vif-deleted', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:49.765267 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-fd19cdd2-e140-40ec-88f9-49dbb4eb3eff tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.42 "GET /v2.0/networks?tenant_id=625c6d6c1b4949ce817bdff51006604f&shared=False HTTP/1.1" status: 200 len: 213 time: 0.3115153 May 25 16:05:49.876797 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-f8cd3921-a574-47aa-b474-3faaf956ecc9 req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] 10.222.0.42 "DELETE /v2.0/ports/11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 HTTP/1.1" status: 204 len: 173 time: 0.9857292 May 25 16:05:50.007600 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-302fa2d1-2349-4ada-9996-c668b9d6111b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:50.008314 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-302fa2d1-2349-4ada-9996-c668b9d6111b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Synchronizing usage tracker for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:05:50.032106 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-68ed493a-3641-4a25-8f94-d390ded2882b tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:50.033550 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-68ed493a-3641-4a25-8f94-d390ded2882b tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 769 time: 0.2571290 May 25 16:05:50.048165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0643b28-4658-4358-8a9f-d0595faa16b0 req-75e4a912-5bf0-428d-9c98-2d3ae5753cc6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:50.048165 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0643b28-4658-4358-8a9f-d0595faa16b0 req-75e4a912-5bf0-428d-9c98-2d3ae5753cc6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.5430543 May 25 16:05:50.083399 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Request body: {'port': {'device_id': '457327e7-4504-4daf-bfea-f8cb1178b775', 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008', 'admin_state_up': True, 'tenant_id': '625c6d6c1b4949ce817bdff51006604f'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:50.184259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-302fa2d1-2349-4ada-9996-c668b9d6111b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Unset dirty status for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:05:50.185409 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-302fa2d1-2349-4ada-9996-c668b9d6111b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/networks?id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f HTTP/1.1" status: 200 len: 813 time: 0.7735927 May 25 16:05:50.196784 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0643b28-4658-4358-8a9f-d0595faa16b0 req-868c5ac4-872c-47f8-94ae-72ad013b3db7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1435626 May 25 16:05:50.213819 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-f89232fc-9612-4aa9-9cf4-07dd22568773 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/quotas/e8bb2885c2ca420b852158159d30e3b4 HTTP/1.1" status: 200 len: 375 time: 0.0210669 May 25 16:05:50.344273 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-48b8d5e4-8d5f-40f0-b904-e47e296c1d51 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&fields=id HTTP/1.1" status: 200 len: 210 time: 0.1176701 May 25 16:05:50.516336 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:50.525571 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Usage tracker for resource:port and tenant:625c6d6c1b4949ce817bdff51006604f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:50.530246 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:50.539193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:50.593143 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-154099c2-895b-4e13-8bc2-b3db7c66443d service neutron] 10.222.0.82 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1872122 May 25 16:05:50.601665 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4eedb232-9f68-4fb3-b631-4e0fac251996 req-487030f0-ea76-4389-9f67-cee013a770db tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:50.603143 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4eedb232-9f68-4fb3-b631-4e0fac251996 req-487030f0-ea76-4389-9f67-cee013a770db tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.2852280 May 25 16:05:50.694069 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Made reservation on behalf of 625c6d6c1b4949ce817bdff51006604f for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:50.697502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:50.730448 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:50.765145 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4eedb232-9f68-4fb3-b631-4e0fac251996 req-05c02207-77ee-4b44-aaf6-fd675118aad1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1429338 May 25 16:05:50.776040 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:50.784060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Usage tracker for resource:security_group_rule and tenant:625c6d6c1b4949ce817bdff51006604f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:50.788944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:50.805695 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:50.814663 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-7b8f5c28-c90d-4fb2-9c1b-229bb4730028 service neutron] 10.222.0.82 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2105577 May 25 16:05:50.871775 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'dce17d11-d553-46f1-9694-cc84fef1b4be', 'name': 'network-vif-deleted', 'tag': '11c6b8f1-f51c-4048-aa6e-456bb22bb7c5', 'status': 'completed', 'code': 200} May 25 16:05:50.920019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:51.031514 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:51.036889 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ce7a99ee-90df-4ca3-915d-e9eb6cab768b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:51.136231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Pushing event updated for resources: {'SecurityGroup': ['ID=71d2df97-673b-4fb7-bedf-e2c7419ef5a7,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:51.148675 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:51.208258 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c427b0e7-9483-45f2-aa48-3906ae23051d req-8384a88a-cf93-4699-9c26-19eac8067ded tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.0606248 May 25 16:05:51.292807 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09adcb8e-a43d-4330-99e1-d59246deeced req-d56f02db-473c-4b19-be0f-9f43dfc1c373 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:51.299973 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-09adcb8e-a43d-4330-99e1-d59246deeced req-d56f02db-473c-4b19-be0f-9f43dfc1c373 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.3135478 May 25 16:05:51.343451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Allocated IP 192.168.233.181 (8153c8a7-e464-4057-a931-9802f12fc008/4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb/990b27e0-71fb-4555-a477-ef91b784cd8a) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:05:51.362230 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-09adcb8e-a43d-4330-99e1-d59246deeced req-67625855-2fe3-4cef-ab58-0f3519a0535c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0613670 May 25 16:05:51.493246 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a2c0ce0-b4fd-41f6-8ccb-8ccfd95cedaf req-f23f8be1-d1b1-4076-87df-7ef618ce2f3a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.0776563 May 25 16:05:51.763127 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:05:51.802387 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:05:51.802742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 11c6b8f1-f51c-4048-aa6e-456bb22bb7c5 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:05:51.822207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:51.848056 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Transition to ACTIVE for port object 990b27e0-71fb-4555-a477-ef91b784cd8a will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:05:51.856493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-700c25ba-3bc7-4558-83df-dde51359162a req-5e328eea-3947-41f7-94d5-c8689dcd928e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:51.858247 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-700c25ba-3bc7-4558-83df-dde51359162a req-5e328eea-3947-41f7-94d5-c8689dcd928e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2450736 May 25 16:05:51.918865 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-700c25ba-3bc7-4558-83df-dde51359162a req-f868f73d-b9c0-4c48-b66f-426b209b17ef tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0545886 May 25 16:05:52.026718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:52.045334 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:52.191646 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Network 8153c8a7-e464-4057-a931-9802f12fc008 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:05:52.198648 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Pushing event updated for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:52.201933 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.157s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:52.206473 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:52.206473 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:05:52.283914 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:52.284504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:52.358811 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0dbc38f-6b52-4eff-9697-5086a1e90ca0 req-58de046f-986b-4cb1-b253-c0fa2f745b78 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:52.360123 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0dbc38f-6b52-4eff-9697-5086a1e90ca0 req-58de046f-986b-4cb1-b253-c0fa2f745b78 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2672570 May 25 16:05:52.447402 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 938 time: 2.3737080 May 25 16:05:52.473970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Request body: {'port': {'device_id': '457327e7-4504-4daf-bfea-f8cb1178b775', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:52.552760 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0dbc38f-6b52-4eff-9697-5086a1e90ca0 req-e03b90db-ee87-4417-94c1-ccf1e305cb68 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1270156 May 25 16:05:52.641529 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-d477b0e3-2aef-4ec3-b8f0-5b063b5c6fa0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:52.642376 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-d477b0e3-2aef-4ec3-b8f0-5b063b5c6fa0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Synchronizing usage tracker for tenant:625c6d6c1b4949ce817bdff51006604f on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:05:52.646155 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:52.652623 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-486df67a-8dad-4005-9a9a-51b594fedbc4 None None] Provisioning for port 990b27e0-71fb-4555-a477-ef91b784cd8a completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:05:52.657717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-486df67a-8dad-4005-9a9a-51b594fedbc4 None None] Provisioning complete for port 990b27e0-71fb-4555-a477-ef91b784cd8a triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:05:52.658089 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-486df67a-8dad-4005-9a9a-51b594fedbc4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:52.715818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-486df67a-8dad-4005-9a9a-51b594fedbc4 None None] Port 990b27e0-71fb-4555-a477-ef91b784cd8a cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:05:52.891763 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3110100a-ad71-45bd-9f3f-d6a5a281df08 req-d376ade8-800b-4410-a577-cdab224df359 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:52.895697 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3110100a-ad71-45bd-9f3f-d6a5a281df08 req-d376ade8-800b-4410-a577-cdab224df359 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2200871 May 25 16:05:52.940086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-d477b0e3-2aef-4ec3-b8f0-5b063b5c6fa0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Unset dirty status for tenant:625c6d6c1b4949ce817bdff51006604f on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:05:52.941226 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-d477b0e3-2aef-4ec3-b8f0-5b063b5c6fa0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 936 time: 0.5798419 May 25 16:05:53.047585 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3110100a-ad71-45bd-9f3f-d6a5a281df08 req-0b4cebd9-fb8d-4939-9264-2ece6731a39f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1456139 May 25 16:05:53.090025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-08d6abaa-7893-4ba5-a5b5-304e76eacbd7 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Synchronizing usage tracker for tenant:625c6d6c1b4949ce817bdff51006604f on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:05:53.260962 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-08d6abaa-7893-4ba5-a5b5-304e76eacbd7 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Unset dirty status for tenant:625c6d6c1b4949ce817bdff51006604f on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:05:53.260962 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10fa5e6a-dac4-4fb2-a2a5-d02f7d864f1a req-08d6abaa-7893-4ba5-a5b5-304e76eacbd7 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2665753 May 25 16:05:53.261275 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5651bfd-607b-4330-b69c-8abb8dbd29e0 req-b0e8a3f4-a2de-4efe-acbb-f07d880034b6 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.0528007 May 25 16:05:53.267873 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 990b27e0-71fb-4555-a477-ef91b784cd8a {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:05:53.284891 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:53.297747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Transition to ACTIVE for port object 990b27e0-71fb-4555-a477-ef91b784cd8a will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:05:53.452339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:53.458448 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:53.657850 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Pushing event updated for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:53.658791 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.200s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:53.714618 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5d1214f-82b6-4a42-a424-16e14dadedae req-4e9c8417-f011-42c6-b561-946723be6937 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:53.716524 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5d1214f-82b6-4a42-a424-16e14dadedae req-4e9c8417-f011-42c6-b561-946723be6937 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.3375704 May 25 16:05:53.791832 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5d1214f-82b6-4a42-a424-16e14dadedae req-5525fe66-5428-47a1-ade2-c2bc23bf37af tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0663831 May 25 16:05:53.801532 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Attempting to bind port 990b27e0-71fb-4555-a477-ef91b784cd8a on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:05:53.801994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Attempting to bind port 990b27e0-71fb-4555-a477-ef91b784cd8a by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:05:53.802240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Attempting to bind port 990b27e0-71fb-4555-a477-ef91b784cd8a on network 8153c8a7-e464-4057-a931-9802f12fc008 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:53.809029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Port 990b27e0-71fb-4555-a477-ef91b784cd8a on network 8153c8a7-e464-4057-a931-9802f12fc008 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:05:53.809029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Attempting to bind port 990b27e0-71fb-4555-a477-ef91b784cd8a on network 8153c8a7-e464-4057-a931-9802f12fc008 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:05:53.819012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 5, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:05:53.819345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Checking segment: {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:05:53.819885 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Bound using segment: {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:05:53.820422 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Bound port: 990b27e0-71fb-4555-a477-ef91b784cd8a, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bdf1b954-6f11-47d3-84b2-de67cb821027', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 926, 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:05:53.820810 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:53.994020 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:05:54.014321 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=990b27e0-71fb-4555-a477-ef91b784cd8a,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:05:54.043476 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ab12d2c8-aeeb-42c6-8774-ae154813efb3 req-170a1161-6866-49d9-a888-8cb458ed1c45 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.048590 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ab12d2c8-aeeb-42c6-8774-ae154813efb3 req-170a1161-6866-49d9-a888-8cb458ed1c45 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1235836 May 25 16:05:54.110257 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ab12d2c8-aeeb-42c6-8774-ae154813efb3 req-9759046e-571a-41fb-8ffc-74df6946dca3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0542579 May 25 16:05:54.138961 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:54.156608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:54.380017 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.380861 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:54.382248 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] 10.222.0.42 "PUT /v2.0/ports/990b27e0-71fb-4555-a477-ef91b784cd8a HTTP/1.1" status: 200 len: 1098 time: 1.9169419 May 25 16:05:54.386008 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:05:54.467912 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Pushing event updated for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:54.471887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.322s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:54.507068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0dff4956-d6a3-46c9-8d55-cc4353fc2aee req-f2d19344-4cae-48bb-ba88-31f332ea0747 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.507068 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0dff4956-d6a3-46c9-8d55-cc4353fc2aee req-f2d19344-4cae-48bb-ba88-31f332ea0747 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1363783 May 25 16:05:54.584083 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0dff4956-d6a3-46c9-8d55-cc4353fc2aee req-6dca079c-f9bb-45b3-9fb3-c7d4e8e238f8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0697074 May 25 16:05:54.702255 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-15050a96-1947-4dc3-89bf-70a3bdf8f4d6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.704126 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-15050a96-1947-4dc3-89bf-70a3bdf8f4d6 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=625c6d6c1b4949ce817bdff51006604f&device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 1101 time: 0.0969713 May 25 16:05:54.938271 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3d373bf2-5cb8-4ed6-8fe6-32beb5276a09 req-941ade42-f74f-488e-ba81-db4db280652f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.1043150 May 25 16:05:54.948117 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a7d1a6cd-25fd-4994-b4e8-ea075b889441 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.949651 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a7d1a6cd-25fd-4994-b4e8-ea075b889441 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "GET /v2.0/networks?id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f HTTP/1.1" status: 200 len: 813 time: 0.3428810 May 25 16:05:54.995982 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a4809611-b442-49aa-b72d-e1a9c70c962b req-6865f741-0a00-4f86-a280-865d8e0cf7c0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:54.997463 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a4809611-b442-49aa-b72d-e1a9c70c962b req-6865f741-0a00-4f86-a280-865d8e0cf7c0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 948 time: 0.1733518 May 25 16:05:55.012264 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a', 'status': 'completed', 'code': 200} May 25 16:05:55.048450 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a4809611-b442-49aa-b72d-e1a9c70c962b req-81878c3d-eb4a-444a-a7fc-718ce7eea6d6 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444419 May 25 16:05:55.215614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-66f27e2a-15cc-4167-961b-662cff408886 req-bf3942cb-cf85-41bb-a654-61bbdb174c44 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:55.217212 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-66f27e2a-15cc-4167-961b-662cff408886 req-bf3942cb-cf85-41bb-a654-61bbdb174c44 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0905023 May 25 16:05:55.317880 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-66f27e2a-15cc-4167-961b-662cff408886 req-56cc29da-3a9c-4456-887e-ae8fe1fc23d9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0938158 May 25 16:05:55.349527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fe93be5-11a5-4fab-9cfb-8a676a6393bb req-b771ee07-645f-4b00-a477-8dd935182216 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:55.352557 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2fe93be5-11a5-4fab-9cfb-8a676a6393bb req-b771ee07-645f-4b00-a477-8dd935182216 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1267471 May 25 16:05:55.422849 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2fe93be5-11a5-4fab-9cfb-8a676a6393bb req-b3e7e87c-44cf-4bf0-b5a8-03d7a5e61604 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0583847 May 25 16:05:55.444251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44490) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:55.587686 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:55.603962 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:55.608052 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Usage tracker for resource:security_group_rule and tenant:c9775e2c78364d14a3e3082ad650064f is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:55.611114 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:55.617807 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:55.719407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:55.784133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:55.793754 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3ab6ffa6-b334-4e68-9de5-53dd1f9f73ec admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:55.802521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c03c30c6-42ab-4df4-92ea-f23112e16020 req-2799b0f3-694b-45e6-abd4-781092092356 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:55.805197 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c03c30c6-42ab-4df4-92ea-f23112e16020 req-2799b0f3-694b-45e6-abd4-781092092356 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.0987575 May 25 16:05:55.836105 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a6f62116-ade3-49f3-ba5b-6ffbdb5cc06b,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:55.840428 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=c9775e2c78364d14a3e3082ad650064f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3940842 May 25 16:05:55.841388 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:55.843350 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44500) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:05:55.860987 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c03c30c6-42ab-4df4-92ea-f23112e16020 req-0ca7a6d2-0a63-4630-a268-030f4ce722f7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0475221 May 25 16:05:55.918903 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:55.952846 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:56.094031 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-af4f6857-ee7e-47d4-a19b-1acc32341282 req-878bf987-87c4-4cb5-9b31-438a569585b0 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.0443304 May 25 16:05:56.116107 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:56.120117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-adecebd2-4057-450e-9ca5-960f1b387414 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:56.132552 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a6f62116-ade3-49f3-ba5b-6ffbdb5cc06b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:56.134756 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:56.141315 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:56.145006 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/a6f62116-ade3-49f3-ba5b-6ffbdb5cc06b HTTP/1.1" status: 204 len: 168 time: 0.2981970 May 25 16:05:56.258397 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95e57ffb-897b-468e-85c0-382766c0493f req-a2acc649-8610-4e69-bf04-246d0282cd8c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:56.261462 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95e57ffb-897b-468e-85c0-382766c0493f req-a2acc649-8610-4e69-bf04-246d0282cd8c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 948 time: 0.0943012 May 25 16:05:56.306278 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95e57ffb-897b-468e-85c0-382766c0493f req-f3446d12-e60a-47f3-b207-8762681f97f0 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0404074 May 25 16:05:56.531491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-876836f0-8231-4402-8769-7ae7c7253cdb req-20c3ee99-2293-4fee-b1a8-5081c4d423c1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:56.533210 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-876836f0-8231-4402-8769-7ae7c7253cdb req-20c3ee99-2293-4fee-b1a8-5081c4d423c1 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0922635 May 25 16:05:56.641514 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-876836f0-8231-4402-8769-7ae7c7253cdb req-1c545504-5469-4d34-bbb9-73c614bb43b4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1008136 May 25 16:05:56.677003 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-420ab59a-fb01-4c54-b34c-6f9b753bae4c req-4b9cc265-3a52-4d0e-8183-3522793a505f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:56.677003 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-420ab59a-fb01-4c54-b34c-6f9b753bae4c req-4b9cc265-3a52-4d0e-8183-3522793a505f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1038520 May 25 16:05:56.723917 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-420ab59a-fb01-4c54-b34c-6f9b753bae4c req-3ffad09d-2e6a-440b-a2a7-0e1bfa866106 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0415928 May 25 16:05:57.074901 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6a270b9-3ffc-44af-a910-cfa9bcb9ccec req-3298a8a9-11eb-4e51-bf54-4a5fd1952356 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:57.076525 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6a270b9-3ffc-44af-a910-cfa9bcb9ccec req-3298a8a9-11eb-4e51-bf54-4a5fd1952356 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.0928400 May 25 16:05:57.129115 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6a270b9-3ffc-44af-a910-cfa9bcb9ccec req-aca2e149-cf6d-4689-ba28-a8a2d5ee8d28 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0468948 May 25 16:05:57.248363 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f2291789-5333-4833-8bde-96445656bf93 req-b61901b6-1e3a-4a11-81d3-703311fe34b7 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.0432987 May 25 16:05:57.417731 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Request body: {'port': {'device_id': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f', 'admin_state_up': True, 'tenant_id': 'e8bb2885c2ca420b852158159d30e3b4'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:57.506605 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-257a701b-7524-4f32-9a0a-d3cffa0bbb9b service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.181&port_id=990b27e0-71fb-4555-a477-ef91b784cd8a HTTP/1.1" status: 200 len: 217 time: 0.1238518 May 25 16:05:57.592140 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-ca4bc2ee-6f68-4d65-bc99-4273b3275969 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:57.596946 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-ca4bc2ee-6f68-4d65-bc99-4273b3275969 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb HTTP/1.1" status: 200 len: 843 time: 0.0827098 May 25 16:05:57.716888 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:05:57.720489 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-10d7c14e-09c5-455c-81c6-404ce06b994a service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:57.722180 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-10d7c14e-09c5-455c-81c6-404ce06b994a service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=8153c8a7-e464-4057-a931-9802f12fc008&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.1186788 May 25 16:05:57.729376 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59519d4a-3c33-4607-aec8-d08443f12f52 req-825772c7-ccb4-41c5-a41e-6486b04fd4da tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:57.737672 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-59519d4a-3c33-4607-aec8-d08443f12f52 req-825772c7-ccb4-41c5-a41e-6486b04fd4da tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 948 time: 0.2888002 May 25 16:05:57.767248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Usage tracker for resource:port and tenant:e8bb2885c2ca420b852158159d30e3b4 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:05:57.781213 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:05:57.792958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:05:57.813338 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-59519d4a-3c33-4607-aec8-d08443f12f52 req-07e0c04c-eb65-4be0-a7e3-2b00b5abd969 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0744178 May 25 16:05:57.914738 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-0cf2e870-e6b6-4a65-a278-7b17e8501cac service neutron] 10.222.0.42 "GET /v2.0/networks/8153c8a7-e464-4057-a931-9802f12fc008?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1847386 May 25 16:05:57.930935 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-df5d23e0-82ba-4d9d-8732-2e923e5638fd req-b31e80b0-2417-413f-98e4-ee7165b8f421 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:57.932748 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-df5d23e0-82ba-4d9d-8732-2e923e5638fd req-b31e80b0-2417-413f-98e4-ee7165b8f421 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1156347 May 25 16:05:57.984061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Made reservation on behalf of e8bb2885c2ca420b852158159d30e3b4 for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:05:57.988108 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:58.139689 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-df5d23e0-82ba-4d9d-8732-2e923e5638fd req-d4ee5787-00a0-4e6d-9bd8-d44b30ee61dc tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1629999 May 25 16:05:58.193698 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-05a10123-98e4-4f17-a3fb-d8cd8947d722 req-c2985d25-11be-4f1d-832c-68d143a48b15 service neutron] 10.222.0.42 "GET /v2.0/networks/8153c8a7-e464-4057-a931-9802f12fc008?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2715888 May 25 16:05:58.198370 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-05a1cba2-8759-482b-accb-7156a8d30fed req-763b05a7-e53c-48ce-ac2c-89625cfe3ffe tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:58.199950 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a1cba2-8759-482b-accb-7156a8d30fed req-763b05a7-e53c-48ce-ac2c-89625cfe3ffe tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2662787 May 25 16:05:58.458083 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-977c4e62-7a45-41b9-b2e5-f8a902c82d9e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:58.459271 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-05a1cba2-8759-482b-accb-7156a8d30fed req-ceba25c5-b287-4273-a512-92d184739705 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1321089 May 25 16:05:58.461133 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Allocated IP 172.20.0.14 (610dbdeb-d88d-45f7-b872-9563bbbe7c3f/2521d219-c255-41fe-80ad-cbccc4e9c1fe/f1cbd3dd-0666-4b74-80e3-36426f4ec54f) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:05:58.468196 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-977c4e62-7a45-41b9-b2e5-f8a902c82d9e service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=625c6d6c1b4949ce817bdff51006604f&device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 1101 time: 0.0940521 May 25 16:05:58.527152 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e42bff18-0520-4a54-af80-1fd3e8a04547 req-2cfcdecc-f180-4122-8e5e-d9f0ea85e992 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:58.535356 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e42bff18-0520-4a54-af80-1fd3e8a04547 req-2cfcdecc-f180-4122-8e5e-d9f0ea85e992 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2070153 May 25 16:05:58.652217 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-51813a05-e469-4ebe-98d3-f5ba975bd657 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:58.653728 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-51813a05-e469-4ebe-98d3-f5ba975bd657 service neutron] 10.222.0.42 "GET /v2.0/networks?id=8153c8a7-e464-4057-a931-9802f12fc008 HTTP/1.1" status: 200 len: 870 time: 0.1791127 May 25 16:05:58.750485 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-92573ccc-f37a-46bb-81fe-a32c1f60017e service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=192.168.233.181&port_id=990b27e0-71fb-4555-a477-ef91b784cd8a HTTP/1.1" status: 200 len: 217 time: 0.0773716 May 25 16:05:58.776182 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e42bff18-0520-4a54-af80-1fd3e8a04547 req-aca54450-f821-42bc-bf33-da80755e218e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2397768 May 25 16:05:58.787201 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6f91b482-c64d-47f1-8619-71bd85d517b7 req-7d19cdd6-16f5-45ad-bbc3-a2b458b63a3a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.1392128 May 25 16:05:58.848704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-6a20e65d-7087-4b27-aac5-362773e466a5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:58.851200 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-6a20e65d-7087-4b27-aac5-362773e466a5 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb HTTP/1.1" status: 200 len: 843 time: 0.0906553 May 25 16:05:58.885011 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:59.000185 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-3d4685ef-2948-4412-96c2-d869ad4b7f19 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:59.001599 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-3d4685ef-2948-4412-96c2-d869ad4b7f19 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=8153c8a7-e464-4057-a931-9802f12fc008&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1178 time: 0.1389670 May 25 16:05:59.015804 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:59.038530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:05:59.147502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:05:59.153260 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:05:59.163577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:05:59.230995 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Pushing event updated for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:05:59.250506 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-126dcc7e-8de4-45d1-9511-80ef164c0093 service neutron] 10.222.0.42 "GET /v2.0/networks/8153c8a7-e464-4057-a931-9802f12fc008?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2060645 May 25 16:05:59.259478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.222s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:05:59.316127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:59.316792 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:59.415203 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4eb1a4e3-b3b9-4859-964c-8c8751820a48 req-30393b87-3d6b-472f-9c1c-d6cf53da99f1 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:59.423637 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4eb1a4e3-b3b9-4859-964c-8c8751820a48 req-30393b87-3d6b-472f-9c1c-d6cf53da99f1 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 948 time: 0.4570980 May 25 16:05:59.489160 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8937d01c-23b0-4ced-8c75-d71e6e86c509 req-2505538d-bbbc-4d4e-8e16-88d5b3b15cbb service neutron] 10.222.0.42 "GET /v2.0/networks/8153c8a7-e464-4057-a931-9802f12fc008?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2131865 May 25 16:05:59.491188 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4eb1a4e3-b3b9-4859-964c-8c8751820a48 req-85ab9be4-41b0-483d-8dcb-8c0cc19b0219 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0657709 May 25 16:05:59.512851 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 2.1086743 May 25 16:05:59.532706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-056fe1d8-f391-4d3c-bd66-ccfe00078404 req-265c7e64-9da4-46bb-a3b2-f6f53e23942d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:59.535717 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-056fe1d8-f391-4d3c-bd66-ccfe00078404 req-265c7e64-9da4-46bb-a3b2-f6f53e23942d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2136073 May 25 16:05:59.542784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Request body: {'port': {'device_id': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:05:59.760631 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-056fe1d8-f391-4d3c-bd66-ccfe00078404 req-d6aea21d-d9d9-4c56-81a2-090ba743ea02 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1595223 May 25 16:05:59.811644 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:05:59.832086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:05:59.867099 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd51a7bc-f024-41e7-9350-a605d3201c6e req-725ba2e2-c206-4506-a027-99844d13b14c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:05:59.869213 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bd51a7bc-f024-41e7-9350-a605d3201c6e req-725ba2e2-c206-4506-a027-99844d13b14c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1924002 May 25 16:05:59.980672 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:05:59.999192 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bd51a7bc-f024-41e7-9350-a605d3201c6e req-ff753e4c-325a-4f0a-8a98-6baf6822a74f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0654240 May 25 16:06:00.003284 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:06:00.098046 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:00.125864 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:06:00.152102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:00.229808 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:00.230836 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 990b27e0-71fb-4555-a477-ef91b784cd8a details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:06:00.287551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:00.378598 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-89fe447e-5cfe-453f-977b-bcba81f55d07 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:00.379504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-89fe447e-5cfe-453f-977b-bcba81f55d07 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Synchronizing usage tracker for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:00.487293 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=6']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:00.493038 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.206s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:00.501117 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1784af25-4685-4eb5-bcfe-c6d8e5a36eca req-355dd887-2bc2-422e-8d7d-76b12d2d7f19 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:00.509548 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1784af25-4685-4eb5-bcfe-c6d8e5a36eca req-355dd887-2bc2-422e-8d7d-76b12d2d7f19 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.2579591 May 25 16:06:00.528370 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=990b27e0-71fb-4555-a477-ef91b784cd8a,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:00.536335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-89fe447e-5cfe-453f-977b-bcba81f55d07 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Unset dirty status for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:00.543980 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-89fe447e-5cfe-453f-977b-bcba81f55d07 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 932 time: 0.5415921 May 25 16:06:00.551247 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '990b27e0-71fb-4555-a477-ef91b784cd8a', 'network_id': '8153c8a7-e464-4057-a931-9802f12fc008', 'port_id': '990b27e0-71fb-4555-a477-ef91b784cd8a', 'mac_address': 'fa:16:3e:44:9f:55', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 926, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '4ec64d2b-c385-40aa-a0ed-afbb2ef8c3bb', 'ip_address': '192.168.233.181'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:06:00.591477 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.db.resource_extend [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] It took 0.11 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 25 16:06:00.630057 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1784af25-4685-4eb5-bcfe-c6d8e5a36eca req-c227927b-613b-4ad8-80a4-cb41cccaea81 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1192496 May 25 16:06:00.681870 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:00.715838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 990b27e0-71fb-4555-a477-ef91b784cd8a {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:06:00.728063 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id f1cbd3dd-0666-4b74-80e3-36426f4ec54f {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:06:00.742194 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:00.749907 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=990b27e0-71fb-4555-a477-ef91b784cd8a,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:00.783966 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Transition to ACTIVE for port object f1cbd3dd-0666-4b74-80e3-36426f4ec54f will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:00.798031 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-d1ca01f6-0df8-479b-8fde-42f4d581df45 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Synchronizing usage tracker for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:00.853022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:00.855963 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:00.974366 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:00.985238 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:01.026671 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:01.055016 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:01.158774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-d1ca01f6-0df8-479b-8fde-42f4d581df45 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Unset dirty status for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:01.160223 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bbe9e1cb-7ed9-4c13-9a86-6db3ae478198 req-d1ca01f6-0df8-479b-8fde-42f4d581df45 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.6151168 May 25 16:06:01.263955 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:06:01.293639 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Pushing event updated for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:01.296891 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.243s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:01.445796 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['5df9097f-ef39-4cbc-b791-fce24729f2ed'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:01.548426 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78c83e59-da36-4abf-90da-fa8cd3ad2575 req-bfdbbee3-0dc8-4808-98f0-7f8a56043666 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:01.551049 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-78c83e59-da36-4abf-90da-fa8cd3ad2575 req-bfdbbee3-0dc8-4808-98f0-7f8a56043666 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 949 time: 0.7392921 May 25 16:06:01.733358 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Attempting to bind port f1cbd3dd-0666-4b74-80e3-36426f4ec54f on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:01.734250 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Attempting to bind port f1cbd3dd-0666-4b74-80e3-36426f4ec54f by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:01.735018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Attempting to bind port f1cbd3dd-0666-4b74-80e3-36426f4ec54f on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:01.767362 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Port f1cbd3dd-0666-4b74-80e3-36426f4ec54f on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:06:01.768259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Attempting to bind port f1cbd3dd-0666-4b74-80e3-36426f4ec54f on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:01.781157 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-78c83e59-da36-4abf-90da-fa8cd3ad2575 req-4429b470-8e9d-491f-b675-e14f1a23eb81 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2059267 May 25 16:06:01.792242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 5, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:01.792769 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Checking segment: {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:01.794899 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Bound using segment: {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:01.794899 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Bound port: f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:01.794899 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:01.851412 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['990b27e0-71fb-4555-a477-ef91b784cd8a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:01.975742 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-47c789f4-87cb-45d4-82d8-f3567c96ac60 req-2f49ccca-92ce-4dfe-909c-15257b1cf684 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:01.978519 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-47c789f4-87cb-45d4-82d8-f3567c96ac60 req-2f49ccca-92ce-4dfe-909c-15257b1cf684 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.5421767 May 25 16:06:02.065683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2ad3fafd-e007-4cdf-8a20-60797afed075 req-8f3c2634-39ba-4c55-867e-9fdcee84ae14 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:02.069263 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2ad3fafd-e007-4cdf-8a20-60797afed075 req-8f3c2634-39ba-4c55-867e-9fdcee84ae14 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.2236669 May 25 16:06:02.195211 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-47c789f4-87cb-45d4-82d8-f3567c96ac60 req-06126ad3-3571-403c-8be0-2e39dd005c33 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1834819 May 25 16:06:02.224854 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:02.237275 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:02.279684 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0eb03978-4e2d-432a-873f-50995f12c6c2 req-d2ebb2b8-fb61-4011-ba9d-546aca2812ae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:02.284688 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0eb03978-4e2d-432a-873f-50995f12c6c2 req-d2ebb2b8-fb61-4011-ba9d-546aca2812ae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1604161 May 25 16:06:02.511837 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:02.537874 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a177bf43-7068-4625-9c1e-b6dd9c741c35 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:02.589600 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0eb03978-4e2d-432a-873f-50995f12c6c2 req-2f25f8a0-96b1-486d-9f45-d5afbf4e0469 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2823336 May 25 16:06:02.609361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:02.609968 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:02.611538 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] 10.222.0.42 "PUT /v2.0/ports/f1cbd3dd-0666-4b74-80e3-36426f4ec54f HTTP/1.1" status: 200 len: 1094 time: 3.0744278 May 25 16:06:02.614682 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:02.659348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Pushing event updated for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:02.665560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:02.680480 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-901c3188-e1cc-45a8-9302-5b79ff8c789a req-cd09ce77-8789-4027-9988-f721dc6d8d44 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:02.687669 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-901c3188-e1cc-45a8-9302-5b79ff8c789a req-cd09ce77-8789-4027-9988-f721dc6d8d44 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.3344564 May 25 16:06:02.754965 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-901c3188-e1cc-45a8-9302-5b79ff8c789a req-69f65826-1155-4ac5-8421-83ac38ac51c8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0619872 May 25 16:06:02.918536 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f', 'status': 'completed', 'code': 200} May 25 16:06:02.983183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a6a8cb4c-121e-4af1-8c4d-1e359d5ef190 req-e9ed2018-432a-4147-b2a9-45d581c27c80 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:02.985048 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a6a8cb4c-121e-4af1-8c4d-1e359d5ef190 req-e9ed2018-432a-4147-b2a9-45d581c27c80 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 944 time: 0.1278641 May 25 16:06:03.089203 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a6a8cb4c-121e-4af1-8c4d-1e359d5ef190 req-10b6553e-51e1-4f28-a7fa-2b2c6204080a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0977252 May 25 16:06:03.205578 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-889e2c4a-5fab-4a6c-ba47-4e8932d756f2 req-68583f00-6968-44dd-a8b3-e8b32d9ed29e tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:03.220454 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-889e2c4a-5fab-4a6c-ba47-4e8932d756f2 req-68583f00-6968-44dd-a8b3-e8b32d9ed29e tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 949 time: 0.1804912 May 25 16:06:03.333708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-889e2c4a-5fab-4a6c-ba47-4e8932d756f2 req-74d378e7-5856-493c-8a70-735c260518cc tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1072991 May 25 16:06:03.537601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc5381e2-a540-4e53-9f6e-267a72cad4b6 req-dc28b5d6-a835-48b1-ba17-ab6e83466526 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:03.540787 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fc5381e2-a540-4e53-9f6e-267a72cad4b6 req-dc28b5d6-a835-48b1-ba17-ab6e83466526 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1937613 May 25 16:06:03.603372 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fc5381e2-a540-4e53-9f6e-267a72cad4b6 req-96db669c-cfb3-4ae1-8cb0-a3b3dc562421 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0544870 May 25 16:06:04.019283 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-110b87f9-7c73-4b9e-93dd-31fa36720a8a req-c54448c3-9933-4a08-96f3-64d0511845e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:04.023936 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-110b87f9-7c73-4b9e-93dd-31fa36720a8a req-c54448c3-9933-4a08-96f3-64d0511845e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1840260 May 25 16:06:04.105633 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-110b87f9-7c73-4b9e-93dd-31fa36720a8a req-7181f882-dda9-4705-b1c0-f62c091fc5f6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0743246 May 25 16:06:04.126858 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bff017b7-678d-4f02-9a51-7664b94e67f8 req-fa9935ad-035c-49c1-a030-41ebd3b130d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:04.128437 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bff017b7-678d-4f02-9a51-7664b94e67f8 req-fa9935ad-035c-49c1-a030-41ebd3b130d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1805341 May 25 16:06:04.210342 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bff017b7-678d-4f02-9a51-7664b94e67f8 req-27c03926-54d1-4b49-b827-e3b0d140519e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0724151 May 25 16:06:04.463307 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-67ba256a-265e-4a9f-8032-150de04b0dd5 req-203afd9c-2108-4783-8b25-fa6f6a57f925 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:04.467476 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-67ba256a-265e-4a9f-8032-150de04b0dd5 req-203afd9c-2108-4783-8b25-fa6f6a57f925 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 944 time: 0.1599791 May 25 16:06:04.628236 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-67ba256a-265e-4a9f-8032-150de04b0dd5 req-331dab9c-09ce-4385-b899-35c6d663f858 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1482949 May 25 16:06:04.752280 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddb5204a-89d9-4132-ba2a-837af1d48103 req-8992db72-b234-46df-b845-16281ca37ca3 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:04.765388 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddb5204a-89d9-4132-ba2a-837af1d48103 req-8992db72-b234-46df-b845-16281ca37ca3 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 949 time: 0.2319498 May 25 16:06:04.951569 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddb5204a-89d9-4132-ba2a-837af1d48103 req-daeeb360-4048-48fa-b187-281609880a85 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1848910 May 25 16:06:04.994283 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9fa14270-4e67-490d-996e-aa8737fcce99 req-3458c5f3-4936-43bb-a90a-8d3fa66d3c01 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:05.002235 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9fa14270-4e67-490d-996e-aa8737fcce99 req-3458c5f3-4936-43bb-a90a-8d3fa66d3c01 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1717892 May 25 16:06:05.063525 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9fa14270-4e67-490d-996e-aa8737fcce99 req-8b748d32-9ccc-4565-968e-76f6654f2cff tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0602794 May 25 16:06:05.379977 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2ad3fafd-e007-4cdf-8a20-60797afed075 req-a6d4d739-77a5-4918-ad96-773941bf0b80 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1143999 May 25 16:06:05.619357 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f7717aa3-2dfa-4625-9a57-0466011a80c7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:05.708057 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-6af9280d-79ac-4e5b-9ebb-b3ebd6068121 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:05.711638 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-6af9280d-79ac-4e5b-9ebb-b3ebd6068121 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 1097 time: 0.3975043 May 25 16:06:05.728371 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-731e32e8-ce6b-4692-a49d-72781ba2236c req-a7f4d6a3-b466-4677-bf91-fe213c1e92cf tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:05.736072 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-731e32e8-ce6b-4692-a49d-72781ba2236c req-a7f4d6a3-b466-4677-bf91-fe213c1e92cf tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.4618595 May 25 16:06:05.760541 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd4164c9-0bce-4d53-afb4-9e595acc44bc req-2dbaae09-10d0-4a1a-94dc-446deb50f70d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:05.762208 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cd4164c9-0bce-4d53-afb4-9e595acc44bc req-2dbaae09-10d0-4a1a-94dc-446deb50f70d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.2772150 May 25 16:06:05.870636 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-731e32e8-ce6b-4692-a49d-72781ba2236c req-4fbaa43f-1c73-4dd6-84ed-876c66a259c1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1330030 May 25 16:06:06.101872 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cd4164c9-0bce-4d53-afb4-9e595acc44bc req-0c7a6a20-84af-49d9-95ff-ce802e879f7f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.3301165 May 25 16:06:06.451803 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-360ab5b4-e57e-41e7-b328-370f2ac5dd30 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:06.453371 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-360ab5b4-e57e-41e7-b328-370f2ac5dd30 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=e8bb2885c2ca420b852158159d30e3b4&shared=False HTTP/1.1" status: 200 len: 914 time: 0.6230743 May 25 16:06:06.641437 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8311e491-799f-4ad5-91c1-fbc253d02a6e req-18e863f1-0d1b-4313-b848-14afadd8b9c1 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:06.643125 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8311e491-799f-4ad5-91c1-fbc253d02a6e req-18e863f1-0d1b-4313-b848-14afadd8b9c1 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 944 time: 0.3708713 May 25 16:06:06.692300 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-7ccd385d-cc3f-44ab-a6c3-a36b6e9f1f05 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:06.694472 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dd7a151e-249b-49a5-bdd9-cc9f414a38f8 req-7ccd385d-cc3f-44ab-a6c3-a36b6e9f1f05 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2288024 May 25 16:06:06.704405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1904cff-33fd-4e21-afc5-1077119b6462 req-cad55898-be90-4a2f-b528-bf7520447cab tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:06.712665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1904cff-33fd-4e21-afc5-1077119b6462 req-cad55898-be90-4a2f-b528-bf7520447cab tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 949 time: 0.3119311 May 25 16:06:06.772453 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8311e491-799f-4ad5-91c1-fbc253d02a6e req-dff715fc-6510-4366-993a-6c29da4dae33 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1132343 May 25 16:06:06.829052 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1904cff-33fd-4e21-afc5-1077119b6462 req-672eacca-9231-457a-9ba3-89ec6662758e tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1150661 May 25 16:06:07.076064 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-b0eab102-e534-459d-80a2-ac79b931d8fa service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:07.079202 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-b0eab102-e534-459d-80a2-ac79b931d8fa service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 1097 time: 0.2482686 May 25 16:06:07.191261 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:07.492205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 5df9097f-ef39-4cbc-b791-fce24729f2ed triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:07.492205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:07.761347 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-1b65a350-9485-4a1e-bbf4-d32ed2493a7c service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.14&port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f HTTP/1.1" status: 200 len: 217 time: 0.6549628 May 25 16:06:07.934529 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:08.023956 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:08.104955 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:08.108138 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-73de1662-9bd2-4390-a70e-0f966852e83d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:08.110759 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-73de1662-9bd2-4390-a70e-0f966852e83d service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2521d219-c255-41fe-80ad-cbccc4e9c1fe HTTP/1.1" status: 200 len: 853 time: 0.3367419 May 25 16:06:08.197936 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a985aee4-d55c-47be-b238-003e8018c7ba req-885a131c-8d8f-4588-b29f-c2e71a5d088a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:08.203536 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a985aee4-d55c-47be-b238-003e8018c7ba req-885a131c-8d8f-4588-b29f-c2e71a5d088a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.9086945 May 25 16:06:08.222743 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-69012ea9-b2e0-41ed-95e0-74044c09a099 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1042120 May 25 16:06:08.267598 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8d12a903-1273-4ef8-a392-ab297fd309a7 req-e5db9c77-cf46-4484-adfa-e0538dfe74b1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:08.267598 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d12a903-1273-4ef8-a392-ab297fd309a7 req-e5db9c77-cf46-4484-adfa-e0538dfe74b1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.9686899 May 25 16:06:08.364218 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:08.378126 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:08.379153 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 990b27e0-71fb-4555-a477-ef91b784cd8a up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:08.416692 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:08.472637 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a985aee4-d55c-47be-b238-003e8018c7ba req-2712335e-8b76-43cf-a300-0b31b159fccf tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2622321 May 25 16:06:08.525505 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-64065e81-75db-4950-b332-56b219609f7d service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.2722721 May 25 16:06:08.668474 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8d12a903-1273-4ef8-a392-ab297fd309a7 req-0407899f-ef2b-4fa3-b13d-9ef9c649d5be tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.3130672 May 25 16:06:08.795502 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-80073cc5-e9a7-4b99-966e-22e437cc21e9 service neutron] 10.222.0.42 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.5637088 May 25 16:06:08.895975 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=7']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:08.915048 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.498s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:08.931646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2eeca34f-f067-407f-b590-35045e38e035 req-73ffca52-dc3c-4c36-9a11-f6d25abc64bc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:08.933594 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-73ffca52-dc3c-4c36-9a11-f6d25abc64bc service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.3639891 May 25 16:06:08.962778 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-260fbde0-c5d2-48ee-ba61-49c9a15b7f97 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:09.145272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:09.172590 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 990b27e0-71fb-4555-a477-ef91b784cd8a completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:09.182228 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 990b27e0-71fb-4555-a477-ef91b784cd8a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:09.183047 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:09.340064 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2eeca34f-f067-407f-b590-35045e38e035 req-bf4afb22-fcbb-4352-a5d9-e0556cd8e695 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:09.341849 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-bf4afb22-fcbb-4352-a5d9-e0556cd8e695 service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.3791175 May 25 16:06:09.512254 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:09.534771 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-d0fe1c82-2865-4493-ae65-619139623a54 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.1763115 May 25 16:06:09.581480 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=990b27e0-71fb-4555-a477-ef91b784cd8a,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:09.658452 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:09.660687 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:09.679098 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-40057f31-fe03-465c-b6cf-4f4659aa464d req-00bec5cf-f1b8-4725-bfd4-2926cbf6b39b service neutron] 10.222.0.42 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.7928448 May 25 16:06:09.787024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2eeca34f-f067-407f-b590-35045e38e035 req-288b1bee-d6bf-473d-86a7-a0953badfc35 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:09.793438 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-288b1bee-d6bf-473d-86a7-a0953badfc35 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.2516725 May 25 16:06:09.797349 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f2432f3-361b-4f21-8c64-1ef9e4ecb301 req-ff4813dc-721e-46c3-a25c-dc90235fad6d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:09.810604 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f2432f3-361b-4f21-8c64-1ef9e4ecb301 req-ff4813dc-721e-46c3-a25c-dc90235fad6d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 1.0336514 May 25 16:06:09.898187 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f2432f3-361b-4f21-8c64-1ef9e4ecb301 req-0312e26f-4a24-483a-bd6b-668c596e0038 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0860083 May 25 16:06:09.955871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5789d1c2-36c9-49ec-8d0c-98b1a9604692 req-e07dae59-316d-43de-af87-a691f3cb6367 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:09.959101 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5789d1c2-36c9-49ec-8d0c-98b1a9604692 req-e07dae59-316d-43de-af87-a691f3cb6367 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 944 time: 0.9353449 May 25 16:06:09.964379 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:09.980538 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-0ee3d1a0-e69e-446d-851d-3a44f9372d87 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1313417 May 25 16:06:09.987611 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.327s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:10.135920 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5789d1c2-36c9-49ec-8d0c-98b1a9604692 req-373f50c8-eaec-43b9-8dcf-08f554b52267 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1693707 May 25 16:06:10.346137 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'code': 200} May 25 16:06:10.404020 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-dbd288fe-3758-4bd5-b9cf-51d598f8b6f8 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4188170 May 25 16:06:10.419989 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ffd6777d-6505-4141-be7d-e6aef37131ae req-0f219416-540d-4c11-8226-e733c97ae948 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:10.428423 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ffd6777d-6505-4141-be7d-e6aef37131ae req-0f219416-540d-4c11-8226-e733c97ae948 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.1114430 May 25 16:06:10.489908 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ffd6777d-6505-4141-be7d-e6aef37131ae req-70dd1747-4466-42c0-8ad1-15c1adf03b57 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0603902 May 25 16:06:10.508703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:10.847735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8875e986-206c-4aaf-a634-d0d4c33c865b req-814a587b-bc28-4d73-9e11-d34555a04791 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:10.850008 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8875e986-206c-4aaf-a634-d0d4c33c865b req-814a587b-bc28-4d73-9e11-d34555a04791 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2930043 May 25 16:06:10.995675 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8875e986-206c-4aaf-a634-d0d4c33c865b req-c50bb3c3-abbb-459e-9f23-0478dc5e06c9 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1382463 May 25 16:06:11.089397 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44786) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:11.119241 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2eeca34f-f067-407f-b590-35045e38e035 req-6f553cd1-bcf6-40fe-8136-6f4e5c455e84 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.7043858 May 25 16:06:11.121614 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:11.388090 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:11.438361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:11.445345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Usage tracker for resource:security_group_rule and tenant:2459143587184a5badedc5d39bd6ff79 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:11.453556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:11.461808 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:11.479915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6212cb0-3216-4625-be94-b5fd1b7bdd18 req-0935b4d9-ad3b-4d5e-85ef-eb4aa552a412 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:11.488438 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6212cb0-3216-4625-be94-b5fd1b7bdd18 req-0935b4d9-ad3b-4d5e-85ef-eb4aa552a412 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2558396 May 25 16:06:11.594014 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6212cb0-3216-4625-be94-b5fd1b7bdd18 req-946e673c-369c-4ea2-bf0e-e8af3b2ab335 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1012287 May 25 16:06:11.781047 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:11.859383 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:11.871658 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8916f960-9d59-47a1-bbe5-dc90e297ed10 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:11.982534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=97d1cabd-da25-4151-b4f0-f496b7f59903,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:11.987793 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:12.221864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44806) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:12.223705 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=2459143587184a5badedc5d39bd6ff79&name=default HTTP/1.1" status: 200 len: 2979 time: 1.1327958 May 25 16:06:12.347817 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:12.629539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:12.672577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2fdff139-fe1c-43ab-a35c-e53cac1115e5 req-b63798f6-c1bb-46dd-9d81-835f98b040aa tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:12.726144 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2fdff139-fe1c-43ab-a35c-e53cac1115e5 req-b63798f6-c1bb-46dd-9d81-835f98b040aa tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.6876578 May 25 16:06:12.750311 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:12.897260 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2fdff139-fe1c-43ab-a35c-e53cac1115e5 req-67851e55-c204-4d14-a3cf-1e6547be6d19 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1695287 May 25 16:06:12.956377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10f9ed38-d1a0-48c3-9820-57aea5c36029 req-bc0a9101-14e7-4442-95a9-3589ece3f9e7 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:12.957911 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10f9ed38-d1a0-48c3-9820-57aea5c36029 req-bc0a9101-14e7-4442-95a9-3589ece3f9e7 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 944 time: 0.4547274 May 25 16:06:13.024423 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10f9ed38-d1a0-48c3-9820-57aea5c36029 req-92c27f39-d1cc-4d33-8dc8-ea169c454c79 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0583253 May 25 16:06:13.097313 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:13.116770 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23dd97d2-8d36-4587-beb5-c706c1bbb573 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:13.164720 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=97d1cabd-da25-4151-b4f0-f496b7f59903,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:13.186826 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.187205 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.187546 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.187895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.191475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.191813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.192352 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.193911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.194481 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.196449 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.197114 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.197719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:13.208680 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:13.257860 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:13.264475 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device f1cbd3dd-0666-4b74-80e3-36426f4ec54f details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:06:13.270232 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/97d1cabd-da25-4151-b4f0-f496b7f59903 HTTP/1.1" status: 204 len: 168 time: 1.0414412 May 25 16:06:13.358168 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bd0384c0-2d86-4397-98da-c8b29712ef93 req-5aa514a2-1cbe-4369-9753-f14e14c6fc3c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:13.376691 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bd0384c0-2d86-4397-98da-c8b29712ef93 req-5aa514a2-1cbe-4369-9753-f14e14c6fc3c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.3056681 May 25 16:06:13.527922 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bd0384c0-2d86-4397-98da-c8b29712ef93 req-fb0b143b-907d-4779-8ab4-f2c16cbff60b tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1448064 May 25 16:06:13.791122 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a', 'code': 200} May 25 16:06:13.912617 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:14.086178 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f', 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f', 'port_id': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f', 'mac_address': 'fa:16:3e:7e:91:9c', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 947, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2521d219-c255-41fe-80ad-cbccc4e9c1fe', 'ip_address': '172.20.0.14'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:06:14.446091 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:14.521913 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35d634a0-c009-47c7-bb6e-f6c0a2923f13 req-e3f11cda-f188-4d8b-8db4-dd87e4f333f3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:14.533466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-35d634a0-c009-47c7-bb6e-f6c0a2923f13 req-e3f11cda-f188-4d8b-8db4-dd87e4f333f3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.4660580 May 25 16:06:14.551492 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id f1cbd3dd-0666-4b74-80e3-36426f4ec54f {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:06:14.574762 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:14.589087 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-7c61b3ab-7f6f-433b-bdbe-4bc0615b5911 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:14.591809 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-7c61b3ab-7f6f-433b-bdbe-4bc0615b5911 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 1098 time: 0.3448455 May 25 16:06:14.652115 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-35d634a0-c009-47c7-bb6e-f6c0a2923f13 req-465188df-889e-49bf-87df-e8f3f0b8dfc6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1175387 May 25 16:06:14.687930 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:14.689609 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-630842fd-e2f6-42f9-9e81-674cb9af96e7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:14.695212 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-518477c9-0fbe-48f9-9b1f-aab886b8234a req-bc7a04ff-502b-4852-8ca0-173cb0f9a0b1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:14.702848 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-518477c9-0fbe-48f9-9b1f-aab886b8234a req-bc7a04ff-502b-4852-8ca0-173cb0f9a0b1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.3635972 May 25 16:06:14.835442 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-518477c9-0fbe-48f9-9b1f-aab886b8234a req-bebead47-789f-449d-bde0-dc3bd29d54f0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1289442 May 25 16:06:14.871637 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:14.886323 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.197s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:14.922994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc2ac315-8c3a-41c3-99ac-bd8476855530 req-c4183b57-ec9d-480d-a0be-72586ff6674c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:14.924647 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc2ac315-8c3a-41c3-99ac-bd8476855530 req-c4183b57-ec9d-480d-a0be-72586ff6674c service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 1103 time: 0.1766293 May 25 16:06:14.965199 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-90b9ab1e-9e4b-4df4-b011-c19b6b9f1032 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:14.967144 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-90b9ab1e-9e4b-4df4-b011-c19b6b9f1032 service neutron] 10.222.0.42 "GET /v2.0/networks?id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 200 len: 919 time: 0.3657732 May 25 16:06:15.112874 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-e489e668-c9ce-4a5a-ae58-3f54dfd7309f service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 200 len: 217 time: 0.1344306 May 25 16:06:15.191184 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['f1cbd3dd-0666-4b74-80e3-36426f4ec54f'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:15.252786 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-1fad4727-c51a-40ef-8ca5-50f289a0daa1 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:15.254328 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-1fad4727-c51a-40ef-8ca5-50f289a0daa1 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 200 len: 858 time: 0.1234615 May 25 16:06:15.381882 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-84a8252c-e315-424b-af66-0cfc126db728 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=feb61d72-e6b4-48c6-bc7c-d61f51987cf9&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.1177125 May 25 16:06:15.393992 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc2ac315-8c3a-41c3-99ac-bd8476855530 req-5f2df9f9-9a25-4637-ae2b-80f50812b07b service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3745933 May 25 16:06:15.479141 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3bbc2b32-a789-4cdd-81ff-79ade5d814de req-b71838c3-9676-4533-a6b4-d5409326669f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:15.496370 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3bbc2b32-a789-4cdd-81ff-79ade5d814de req-b71838c3-9676-4533-a6b4-d5409326669f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.5706263 May 25 16:06:15.547632 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-916fd26b-83e8-434c-8982-98c4e279674f req-ac3f4701-6461-4dae-9a08-56cc55a76e5f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:15.571365 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-916fd26b-83e8-434c-8982-98c4e279674f req-ac3f4701-6461-4dae-9a08-56cc55a76e5f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 945 time: 0.4764185 May 25 16:06:15.698642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3bbc2b32-a789-4cdd-81ff-79ade5d814de req-c598650d-d324-4b4f-8bc3-6651b4a4bc60 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2009139 May 25 16:06:15.751990 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-916fd26b-83e8-434c-8982-98c4e279674f req-f88f643a-1511-444d-9786-e82990a61923 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1785960 May 25 16:06:16.108059 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5745aa9e-1ba1-4cc3-b68a-6f460ea48c10 req-b415f71b-fc90-408b-824d-40e32e2a9a89 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:16.111105 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5745aa9e-1ba1-4cc3-b68a-6f460ea48c10 req-b415f71b-fc90-408b-824d-40e32e2a9a89 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2012079 May 25 16:06:16.129706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-53822645-6842-4a77-b3d6-5993eb41d812 req-5fe989e5-073c-4487-a4c8-360dcbca367c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:16.140485 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-53822645-6842-4a77-b3d6-5993eb41d812 req-5fe989e5-073c-4487-a4c8-360dcbca367c tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.1788812 May 25 16:06:16.259155 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44866) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:16.287195 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-53822645-6842-4a77-b3d6-5993eb41d812 req-9dae8348-95c7-4545-ab2c-bedd9b39c9e9 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1450288 May 25 16:06:16.419004 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5745aa9e-1ba1-4cc3-b68a-6f460ea48c10 req-b5de2d6d-3112-4a2b-9949-6e9567f80916 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2129388 May 25 16:06:16.528720 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-57049d97-d711-4fa8-aa13-af81465e6110 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.7548213 May 25 16:06:16.603630 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-1030692280-network', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:16.614330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:16.631165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Usage tracker for resource:network and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:16.640685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:16.705044 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:16.783514 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a93b1dd-ca8f-4a4a-b82b-8e3202720b07 req-d91daf75-d940-4f6b-b6df-b09b39512064 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:16.802828 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a93b1dd-ca8f-4a4a-b82b-8e3202720b07 req-d91daf75-d940-4f6b-b6df-b09b39512064 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1560714 May 25 16:06:16.860094 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Made reservation on behalf of 5d4b9c0f27af44a9ace8862a2133a0cb for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:06:16.868178 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:16.898046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:16.930594 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9ab9d008-35d5-4e61-a304-4c8b3434eb7a req-d570963c-f723-421f-954c-7b5c1ae01d25 service neutron] 10.222.0.42 "GET /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3916931 May 25 16:06:16.946641 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a93b1dd-ca8f-4a4a-b82b-8e3202720b07 req-3f0f5451-73ac-4bba-ad87-693837921254 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1344793 May 25 16:06:16.961175 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:16.973428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:16.982007 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:16.999812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:17.259375 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device f1cbd3dd-0666-4b74-80e3-36426f4ec54f up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:17.340342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:17.520814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:17.539288 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] vlan segment allocate from pool success with {'vlan_id': 920, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:06:17.563420 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:17.563774 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Added segment 79c416e6-aabc-4638-95ed-e2e21a7b289f of type vlan for network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 May 25 16:06:17.604614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:17.635620 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port f1cbd3dd-0666-4b74-80e3-36426f4ec54f completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:17.645429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port f1cbd3dd-0666-4b74-80e3-36426f4ec54f triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:17.645429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:17.652175 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:17.715635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f1048f85-f0d1-40c6-9d99-c667080ff7b6,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:17.746267 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.746634 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.747029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.747390 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.747745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.748093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.748447 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.766393 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.766920 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.767289 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.767652 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:17.768008 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:18.111147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.446s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:18.111147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:18.111147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:18.111147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:18.113717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:18.228058 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:18.291319 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:18.328171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:18.520948 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 882 time: 2.2314870 May 25 16:06:18.526106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 44888) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:18.534959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Request body: {'subnet': {'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-1030692280-subnet', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:18.569741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-092481ed-7fe2-44ae-b676-1890a250b0b0 req-ff6e32a4-8b3a-431a-8a5e-f3a0170fb5ce tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:18.572717 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-092481ed-7fe2-44ae-b676-1890a250b0b0 req-ff6e32a4-8b3a-431a-8a5e-f3a0170fb5ce tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.7932975 May 25 16:06:18.576438 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:18.584252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Pushing event updated for resources: {'Network': ['ID=6c217a71-5586-47ee-ab1e-21c28b2d40a1,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:18.595144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.265s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:18.599434 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:06:18.609218 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Usage tracker for resource:subnet and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:18.628348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:18.660457 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:18.755692 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5eeddd38-1c17-477a-837b-b17c3ed9e119 req-81082613-94d6-4b86-a4c1-b51deee67824 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:18.760950 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5eeddd38-1c17-477a-837b-b17c3ed9e119 req-81082613-94d6-4b86-a4c1-b51deee67824 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 945 time: 0.6620111 May 25 16:06:18.827030 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-092481ed-7fe2-44ae-b676-1890a250b0b0 req-869a56d1-59d1-42ea-8967-c7fae2ddd074 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2104611 May 25 16:06:18.920819 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:18.924906 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5eeddd38-1c17-477a-837b-b17c3ed9e119 req-48cec89a-b7d6-472d-b7b4-81f6b04f911a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1470289 May 25 16:06:18.931790 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:19.035778 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Made reservation on behalf of 5d4b9c0f27af44a9ace8862a2133a0cb for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:06:19.040856 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:19.196111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:19.201581 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.270s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:19.334124 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-937f5588-99e6-498c-9efe-c212cdaca9ad req-6b73d890-ff71-488d-a59b-46c57b2f7f40 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:19.337402 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-937f5588-99e6-498c-9efe-c212cdaca9ad req-6b73d890-ff71-488d-a59b-46c57b2f7f40 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.2090678 May 25 16:06:19.467116 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-937f5588-99e6-498c-9efe-c212cdaca9ad req-9ed9376d-ca42-4e1b-95a3-ce307a35db01 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0834682 May 25 16:06:19.558565 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1995fb81-7962-4895-a5c8-46bd47184c52 req-f9f3a49e-ac00-4cfd-b3df-9147ea2078e9 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:19.569440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1995fb81-7962-4895-a5c8-46bd47184c52 req-f9f3a49e-ac00-4cfd-b3df-9147ea2078e9 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.2306218 May 25 16:06:19.665518 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1995fb81-7962-4895-a5c8-46bd47184c52 req-b029c6b7-b1a3-4ac1-8648-bfa7a5861383 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0945613 May 25 16:06:20.154622 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4b93cd5e-308a-47e9-8073-c487bdd4fa38 req-60c8b824-9fcd-4623-9648-fa6a78c0d0a3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:20.178305 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4b93cd5e-308a-47e9-8073-c487bdd4fa38 req-60c8b824-9fcd-4623-9648-fa6a78c0d0a3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.3011792 May 25 16:06:20.229516 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f4923b6-648f-4400-ad95-0027cbfc5e6b req-180241ad-4f30-4c1a-9480-09f6c6e3adf6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:20.236377 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f4923b6-648f-4400-ad95-0027cbfc5e6b req-180241ad-4f30-4c1a-9480-09f6c6e3adf6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.5325243 May 25 16:06:20.277173 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bc6f0b22-2ccf-4ffb-a35e-b3541a0595c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:20.278007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bc6f0b22-2ccf-4ffb-a35e-b3541a0595c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:20.368911 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4b93cd5e-308a-47e9-8073-c487bdd4fa38 req-8d23b56b-5c0a-4ee7-8ab9-d008b36cc33b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1895590 May 25 16:06:20.373572 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f4923b6-648f-4400-ad95-0027cbfc5e6b req-11bfee40-b43d-47e8-8e9e-9c00623c8f96 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1297941 May 25 16:06:20.409806 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:20.417994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:20.514911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Pushing event updated for resources: {'Subnet': ['ID=2b5e1fe5-3648-40fc-a046-12baa71ed42b,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:20.520500 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bc6f0b22-2ccf-4ffb-a35e-b3541a0595c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:20.523107 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bc6f0b22-2ccf-4ffb-a35e-b3541a0595c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 946 time: 0.5634263 May 25 16:06:20.524699 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f', 'code': 200} May 25 16:06:20.534140 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:20.993157 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Checking segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:20.993771 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Network hostable DHCP agents. Network: 6c217a71-5586-47ee-ab1e-21c28b2d40a1, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:06:21.122157 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Deleting port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:21.122839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:21.204761 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1b6e16e-996e-491d-a22a-3be346817337 req-ad688019-9741-4936-9db2-7d602e4d7554 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:21.209226 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1b6e16e-996e-491d-a22a-3be346817337 req-ad688019-9741-4936-9db2-7d602e4d7554 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2550786 May 25 16:06:21.297562 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1b6e16e-996e-491d-a22a-3be346817337 req-e67c29f9-565a-4ecc-92de-d6e0609835bb tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0803277 May 25 16:06:21.312224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:06:21.529080 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 6c217a71-5586-47ee-ab1e-21c28b2d40a1) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:21.539110 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5c88b885-f9e9-434c-b56a-28f87015e494 None None] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:06:21.560345 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:21.560954 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:21.745245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c8b82f22-f21a-4181-8114-a23ba286bdf8 req-4418f2f8-49ac-4a6a-8e76-e75ec1b6695d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:21.748475 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c8b82f22-f21a-4181-8114-a23ba286bdf8 req-4418f2f8-49ac-4a6a-8e76-e75ec1b6695d service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 1099 time: 0.3258672 May 25 16:06:21.825455 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] For port 4a1a8795-a2e2-4bdb-a937-fc10b821e188, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4a1a8795-a2e2-4bdb-a937-fc10b821e188,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:21.827030 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:21.980412 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:06:21.982022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:22.040189 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 845 time: 3.5123327 May 25 16:06:22.073134 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 44920) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:22.104990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:22.133779 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:22.145261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Usage tracker for resource:router and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:22.149459 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:22.193250 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:22.280374 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Calling delete_port for 4a1a8795-a2e2-4bdb-a937-fc10b821e188 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:22.306919 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c8b82f22-f21a-4181-8114-a23ba286bdf8 req-ab373261-4bd5-4403-a7e1-7c3854fba2f9 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.3233020 May 25 16:06:22.321812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:22.323491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:22.355338 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:22.434133 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a1910610-ee83-45f3-902f-5070b7db2df5 req-47252149-e029-4933-b473-bc5d9f32515c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:22.449668 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1910610-ee83-45f3-902f-5070b7db2df5 req-47252149-e029-4933-b473-bc5d9f32515c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 0.6972089 May 25 16:06:22.456621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Allocated IP 172.20.0.2 (6c217a71-5586-47ee-ab1e-21c28b2d40a1/2b5e1fe5-3648-40fc-a046-12baa71ed42b/b5e082ba-8608-48eb-ba90-ac1afb9263ff) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:22.571334 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a1910610-ee83-45f3-902f-5070b7db2df5 req-cb1b3807-a60d-4900-9d0d-fa13b9a6dc82 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1171017 May 25 16:06:22.819717 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:22.834969 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d3dbdbf2-83f1-48a9-af0f-d12e23a18080 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:22.923196 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc', '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', '2206a32c-5500-4208-9123-7aab17fc53c7'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:22.998091 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event deleted for resources: {'Port': ['ID=4a1a8795-a2e2-4bdb-a937-fc10b821e188,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:23.096669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.262s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:23.261097 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] No DVR routers for this DVR port 4a1a8795-a2e2-4bdb-a937-fc10b821e188 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:23.273992 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:23.287842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'name': 'network-vif-deleted', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:23.302766 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4a1a8795-a2e2-4bdb-a937-fc10b821e188 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:23.350993 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-56e9b84a-f9e5-49a1-8097-812f1da5f753 req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "DELETE /v2.0/ports/4a1a8795-a2e2-4bdb-a937-fc10b821e188 HTTP/1.1" status: 204 len: 173 time: 2.7773368 May 25 16:06:23.368180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 4a1a8795-a2e2-4bdb-a937-fc10b821e188 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:23.368535 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4a1a8795-a2e2-4bdb-a937-fc10b821e188 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:23.452814 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:23.488681 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Transition to ACTIVE for port object b5e082ba-8608-48eb-ba90-ac1afb9263ff will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:23.685673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Transition to ACTIVE for port object b5e082ba-8608-48eb-ba90-ac1afb9263ff will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:23.811125 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:23.820840 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:23.832924 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-552b9533-49f5-4cb2-a123-bd458416b4ad req-2ef36f70-146b-49f2-a8f9-a0c7fb9eedc1 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=2e9c509c-41ab-401c-a248-d074d815b353 HTTP/1.1" status: 200 len: 210 time: 1.0467665 May 25 16:06:23.977708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:24.008912 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:24.033403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:24.041179 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:24.128420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-80145b36-0242-4722-9c4f-03e17810f4e7 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:24.133560 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-80145b36-0242-4722-9c4f-03e17810f4e7 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.42 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.3334346 May 25 16:06:24.228900 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e40a79f7-1079-460f-b7f1-843138b7fb26 req-dc41f0c5-e8ff-4eed-a33a-ad23a70ee681 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:24.236182 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e40a79f7-1079-460f-b7f1-843138b7fb26 req-dc41f0c5-e8ff-4eed-a33a-ad23a70ee681 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 1.2992024 May 25 16:06:24.269945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-caa9b0a8-5242-4480-9ef2-82ff596c9068 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:24.275015 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-caa9b0a8-5242-4480-9ef2-82ff596c9068 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1098 time: 1.0058541 May 25 16:06:24.557932 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Pushing event updated for resources: {'Port': ['ID=b5e082ba-8608-48eb-ba90-ac1afb9263ff,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:24.602764 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.782s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:24.698357 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Deleting port 990b27e0-71fb-4555-a477-ef91b784cd8a {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:24.698687 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:24.721246 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e40a79f7-1079-460f-b7f1-843138b7fb26 req-7af6ab7b-5256-4712-9890-cef6ece1d40b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.4839516 May 25 16:06:24.765177 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Attempting to bind port b5e082ba-8608-48eb-ba90-ac1afb9263ff on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:24.788015 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Attempting to bind port b5e082ba-8608-48eb-ba90-ac1afb9263ff by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:24.794111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Attempting to bind port b5e082ba-8608-48eb-ba90-ac1afb9263ff on network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:25.022273 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-1c0a1e2d-d117-40ef-ba4e-dc93f81e5dda service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:25.023854 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-1c0a1e2d-d117-40ef-ba4e-dc93f81e5dda service neutron] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 912 time: 0.7176795 May 25 16:06:25.034485 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:25.035126 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Checking segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:25.035555 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:25.035987 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Bound using segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:25.039228 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Bound port: b5e082ba-8608-48eb-ba90-ac1afb9263ff, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:25.039713 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:25.260247 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] For port b5e082ba-8608-48eb-ba90-ac1afb9263ff, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:25.301305 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] For port 990b27e0-71fb-4555-a477-ef91b784cd8a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=990b27e0-71fb-4555-a477-ef91b784cd8a,segment=NetworkSegment(bdf1b954-6f11-47d3-84b2-de67cb821027),segment_id=bdf1b954-6f11-47d3-84b2-de67cb821027)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:25.302481 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:25.333410 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-df2ab307-e885-4a5e-8e21-86c083ed6614 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 1001 time: 0.2987847 May 25 16:06:25.338362 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] For port b5e082ba-8608-48eb-ba90-ac1afb9263ff, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b5e082ba-8608-48eb-ba90-ac1afb9263ff,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:25.544225 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:25.559151 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:25.645355 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Calling delete_port for 990b27e0-71fb-4555-a477-ef91b784cd8a owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:25.646525 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-744916a0-8b5d-46a7-9474-e8287c7dba24 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:25.649988 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-744916a0-8b5d-46a7-9474-e8287c7dba24 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.2930505 May 25 16:06:25.765351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Pushing event updated for resources: {'Port': ['ID=b5e082ba-8608-48eb-ba90-ac1afb9263ff,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:25.779197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.220s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:26.050294 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ff95790-0acd-4dc0-99a5-9ea36d186bda req-97851e60-694e-4dee-aca0-f1a4da0f6d3b tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:26.052698 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ff95790-0acd-4dc0-99a5-9ea36d186bda req-97851e60-694e-4dee-aca0-f1a4da0f6d3b tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 950 time: 0.8980522 May 25 16:06:26.062263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d4cbbf7-d60d-49e2-a81d-cf04c2fd81a5 req-a2964fd0-af1c-40a7-93da-af9b595508d8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:26.064141 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6d4cbbf7-d60d-49e2-a81d-cf04c2fd81a5 req-a2964fd0-af1c-40a7-93da-af9b595508d8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.6080856 May 25 16:06:26.253029 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ff95790-0acd-4dc0-99a5-9ea36d186bda req-cd6f0147-679d-4775-9c04-40821f049a3d tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/security-groups?id=71d2df97-673b-4fb7-bedf-e2c7419ef5a7&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1877880 May 25 16:06:26.300678 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6d4cbbf7-d60d-49e2-a81d-cf04c2fd81a5 req-4536bb69-d10f-44a3-a6c5-2778575e0696 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1977165 May 25 16:06:26.301630 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:26.359320 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-5bfda3a4-08ec-46fa-99da-ce69e35d9356 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:26.362537 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '2e9c509c-41ab-401c-a248-d074d815b353', 'name': 'network-vif-deleted', 'tag': '4a1a8795-a2e2-4bdb-a937-fc10b821e188', 'status': 'completed', 'code': 200} May 25 16:06:26.363140 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-5bfda3a4-08ec-46fa-99da-ce69e35d9356 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.6969795 May 25 16:06:26.365103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:26.578200 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:26.585283 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:26.673806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Allocated IP 172.27.2.159 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/bb4cb419-146e-473c-abad-9dffa503b894) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:26.699944 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Pushing event deleted for resources: {'Port': ['ID=990b27e0-71fb-4555-a477-ef91b784cd8a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:26.718593 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:27.073179 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] No DVR routers for this DVR port 990b27e0-71fb-4555-a477-ef91b784cd8a on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:27.093372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:27.241537 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e87bbc5e-0687-4cf8-acc6-c5bd3dbf6402 req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.42 "DELETE /v2.0/ports/990b27e0-71fb-4555-a477-ef91b784cd8a HTTP/1.1" status: 204 len: 173 time: 3.0998080 May 25 16:06:27.315357 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 990b27e0-71fb-4555-a477-ef91b784cd8a no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:27.374099 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:27.481409 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 990b27e0-71fb-4555-a477-ef91b784cd8a {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:27.482105 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 990b27e0-71fb-4555-a477-ef91b784cd8a not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:27.684873 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e9d2e65f-125d-45c8-9418-306bbfaa5e34 None None] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:06:27.810438 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b5e082ba-8608-48eb-ba90-ac1afb9263ff up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:27.928949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:27.969469 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:28.061996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Pushing event updated for resources: {'Port': ['ID=bb4cb419-146e-473c-abad-9dffa503b894,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:28.065715 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:28.068242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:28.076137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:28.083843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:28.153649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddc5bc8f-c6ee-4326-a558-864138d57c56 req-8cb7e686-d284-449f-9630-4de80cf10563 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:28.155243 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddc5bc8f-c6ee-4326-a558-864138d57c56 req-8cb7e686-d284-449f-9630-4de80cf10563 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 1.0947571 May 25 16:06:28.299185 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-2f204fdc-7577-4c2a-b0be-5b1b8d31bd1c service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 1.8125112 May 25 16:06:28.320833 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-31830c4d-624f-403f-b7c6-27e12efba86c req-6b396593-9527-45a4-bd38-7f25168e6143 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:28.322358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31830c4d-624f-403f-b7c6-27e12efba86c req-6b396593-9527-45a4-bd38-7f25168e6143 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 1.3972471 May 25 16:06:28.363513 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'name': 'network-vif-deleted', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:28.472548 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-d10a2192-8e84-43be-938f-732f22d9a2c2 None None] Provisioning for port b5e082ba-8608-48eb-ba90-ac1afb9263ff completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:28.549607 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port b5e082ba-8608-48eb-ba90-ac1afb9263ff completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:28.555929 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port b5e082ba-8608-48eb-ba90-ac1afb9263ff triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:28.556429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:28.608702 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddc5bc8f-c6ee-4326-a558-864138d57c56 req-5f81f775-99bd-4ae0-bfc9-4a6ae3104fde tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.4076560 May 25 16:06:28.651080 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-31830c4d-624f-403f-b7c6-27e12efba86c req-c20aac67-364a-4b36-945c-2445aa06934b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.3009756 May 25 16:06:28.669296 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-b7394a02-97ef-4591-9f5e-bb304c6d3a8a None None] Provisioning complete for port b5e082ba-8608-48eb-ba90-ac1afb9263ff triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:28.669702 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b7394a02-97ef-4591-9f5e-bb304c6d3a8a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:28.788096 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77c77368-6aa9-42b9-ab13-c78a53ce2a84 req-39327d3f-acd5-4c20-b3d7-3f9d647c68e9 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.4755981 May 25 16:06:28.839613 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:29.030204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port b5e082ba-8608-48eb-ba90-ac1afb9263ff, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b5e082ba-8608-48eb-ba90-ac1afb9263ff,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:29.041268 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b7394a02-97ef-4591-9f5e-bb304c6d3a8a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:29.068810 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a103ed29-5894-4a2b-9455-e895e395cd91 req-197e7352-82d8-4398-bcda-edfa61d53aba tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] 10.222.0.82 "GET /v2.0/ports?device_id=457327e7-4504-4daf-bfea-f8cb1178b775 HTTP/1.1" status: 200 len: 210 time: 0.1320097 May 25 16:06:29.203538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:29.279288 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-45c2012b-8a04-4deb-a888-20ecb780df32 req-bb610e78-d2a5-48d9-aa91-43e5ee9a9ffd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:29.281525 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-45c2012b-8a04-4deb-a888-20ecb780df32 req-bb610e78-d2a5-48d9-aa91-43e5ee9a9ffd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2812979 May 25 16:06:29.282350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:29.325465 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:29.523258 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-b7394a02-97ef-4591-9f5e-bb304c6d3a8a None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 16:06:29.524350 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-b7394a02-97ef-4591-9f5e-bb304c6d3a8a None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 16:06:29.547502 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=b5e082ba-8608-48eb-ba90-ac1afb9263ff,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:29.555696 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.230s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:29.664156 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-45c2012b-8a04-4deb-a888-20ecb780df32 req-f0533163-d822-4a84-bbf6-5a6fc570e642 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2872889 May 25 16:06:29.785115 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '457327e7-4504-4daf-bfea-f8cb1178b775', 'name': 'network-vif-deleted', 'tag': '990b27e0-71fb-4555-a477-ef91b784cd8a', 'status': 'completed', 'code': 200} May 25 16:06:30.285793 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:30.408558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0dfff09-ee90-4aae-86c6-c0e9aa476e13 req-327f1221-d161-4625-8bc4-eee192946302 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:30.416815 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0dfff09-ee90-4aae-86c6-c0e9aa476e13 req-327f1221-d161-4625-8bc4-eee192946302 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.3966877 May 25 16:06:30.665095 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0dfff09-ee90-4aae-86c6-c0e9aa476e13 req-662305b1-0d2c-47a7-a58f-2bbf00714af3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2443049 May 25 16:06:30.724244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6946f275-ef89-47f5-99a5-f718ba52896b req-7a5958d3-8760-4379-ac1f-5d4b1eada8ed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:30.725780 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6946f275-ef89-47f5-99a5-f718ba52896b req-7a5958d3-8760-4379-ac1f-5d4b1eada8ed tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.2890229 May 25 16:06:30.865502 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6946f275-ef89-47f5-99a5-f718ba52896b req-c8c35ad6-6815-4df3-92ac-1bc8e0c5e492 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1033645 May 25 16:06:30.933609 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-caf7234c-c57d-412f-ad44-613007954ec9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:30.939151 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-caf7234c-c57d-412f-ad44-613007954ec9 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1099 time: 0.2215049 May 25 16:06:31.054987 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Router ac04f176-2f32-43dd-b873-426ed50b356d is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99545) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:06:31.217712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf175dd9-e632-4d23-9179-e60f672d2eda req-424f6db5-d8ad-4fff-a3b6-89e1ff12680b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:31.219442 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf175dd9-e632-4d23-9179-e60f672d2eda req-424f6db5-d8ad-4fff-a3b6-89e1ff12680b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.3303471 May 25 16:06:31.228105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-d369d5d4-d241-4816-8a4d-84db699e31c7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:31.228105 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-d369d5d4-d241-4816-8a4d-84db699e31c7 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.2781084 May 25 16:06:31.456108 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:06:31.459998 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-88f7f637-b712-4224-85b0-9922df7f04f0 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1002 time: 0.1790016 May 25 16:06:31.469610 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf175dd9-e632-4d23-9179-e60f672d2eda req-6f1048ef-0b21-463b-9c83-e8bd65100786 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2420621 May 25 16:06:31.476353 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Sync routers for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:06:31.506039 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:31.507617 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:31.582174 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-361b306c-9ee2-476e-bf60-6a4f2ad3d5c7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:31.584386 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-361b306c-9ee2-476e-bf60-6a4f2ad3d5c7 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.1012235 May 25 16:06:31.605607 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b5e082ba-8608-48eb-ba90-ac1afb9263ff up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:31.755337 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:31.755337 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 921 time: 9.6651278 May 25 16:06:31.756514 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45078) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:31.776095 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port b5e082ba-8608-48eb-ba90-ac1afb9263ff triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:31.776490 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:31.869509 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-3c447a9d-a387-4a63-b93e-da464bb62da4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:31.871481 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-3c447a9d-a387-4a63-b93e-da464bb62da4 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2370768 May 25 16:06:32.141401 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:32.319490 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-1744b135-5011-401c-85f5-efc6cf07b39f service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.4407475 May 25 16:06:32.638577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf57ab5b-772d-4d02-9fba-995fe02d8a6c req-1564dbd6-0ce3-4fa0-95ec-e67a84af4744 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:32.640812 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf57ab5b-772d-4d02-9fba-995fe02d8a6c req-1564dbd6-0ce3-4fa0-95ec-e67a84af4744 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.5520542 May 25 16:06:32.644852 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-016c1ebe-c057-4859-816b-1f93d5d9f114 req-ee901fef-40d7-4307-9889-a26ba2d3c56b service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2653813 May 25 16:06:32.735915 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf57ab5b-772d-4d02-9fba-995fe02d8a6c req-7bb05941-db82-47a4-bb74-f943a261bb5e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0844665 May 25 16:06:32.854601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-328f0c81-68c9-4c46-8047-b71b4bb2168d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:32.855483 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-328f0c81-68c9-4c46-8047-b71b4bb2168d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:33.000756 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-328f0c81-68c9-4c46-8047-b71b4bb2168d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:33.001877 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-328f0c81-68c9-4c46-8047-b71b4bb2168d tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.4593282 May 25 16:06:33.031443 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:28Z', 'revision_number': 3, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:26Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:33.047032 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Checking router: ac04f176-2f32-43dd-b873-426ed50b356d for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:06:33.048076 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Updating router ac04f176-2f32-43dd-b873-426ed50b356d port bb4cb419-146e-473c-abad-9dffa503b894 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:06:33.166496 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.321906 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f68065e1-718d-41b9-a789-5c94e8293141 req-6783f48b-0a85-45e8-b2bf-dad677789f6e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:33.327623 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f68065e1-718d-41b9-a789-5c94e8293141 req-6783f48b-0a85-45e8-b2bf-dad677789f6e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.3135364 May 25 16:06:33.358452 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fc6c0c6b-8176-4454-add6-66e864cc72f6 req-f50db84d-6329-40b7-b35a-737f986fe5f4 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:33.360159 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fc6c0c6b-8176-4454-add6-66e864cc72f6 req-f50db84d-6329-40b7-b35a-737f986fe5f4 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 945 time: 0.3936231 May 25 16:06:33.411686 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Deleting port 2206a32c-5500-4208-9123-7aab17fc53c7 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:33.420660 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.474933 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f68065e1-718d-41b9-a789-5c94e8293141 req-6bd20b23-2c3d-4934-9f5d-839128c665a6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1437168 May 25 16:06:33.488827 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fc6c0c6b-8176-4454-add6-66e864cc72f6 req-640bc8b6-e88b-488d-8bd7-79ab2b8269ee tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1129422 May 25 16:06:33.750078 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.765842 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Transition to ACTIVE for port object bb4cb419-146e-473c-abad-9dffa503b894 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:33.805987 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] For port 2206a32c-5500-4208-9123-7aab17fc53c7, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2206a32c-5500-4208-9123-7aab17fc53c7,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:33.806589 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.831646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.868585 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:33.877858 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:34.005618 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Pushing event updated for resources: {'Port': ['ID=bb4cb419-146e-473c-abad-9dffa503b894,revision_number=2']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:34.011457 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Attempting to bind port bb4cb419-146e-473c-abad-9dffa503b894 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:34.011995 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Attempting to bind port bb4cb419-146e-473c-abad-9dffa503b894 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:34.012383 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Attempting to bind port bb4cb419-146e-473c-abad-9dffa503b894 on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:34.023495 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.024319 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.024670 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.025014 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.025408 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.025744 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.026086 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.027449 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.027449 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.027449 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.027449 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.028191 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:34.051072 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:34.052213 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cbc37bbe-cc18-4a6c-a54e-2ceb8daffbf1 req-1eb1c2c0-1670-4503-934c-846e3ad77877 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:34.052366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:34.052366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:34.052366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:34.052366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Bound port: bb4cb419-146e-473c-abad-9dffa503b894, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:34.052366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:34.052815 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cbc37bbe-cc18-4a6c-a54e-2ceb8daffbf1 req-1eb1c2c0-1670-4503-934c-846e3ad77877 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.1319253 May 25 16:06:34.056919 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Calling delete_port for 2206a32c-5500-4208-9123-7aab17fc53c7 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:34.090411 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.213s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:34.138255 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cbc37bbe-cc18-4a6c-a54e-2ceb8daffbf1 req-769aebfa-b615-44cc-a350-aeb9a2302372 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0691192 May 25 16:06:34.182504 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] For port bb4cb419-146e-473c-abad-9dffa503b894, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:34.188658 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] For port bb4cb419-146e-473c-abad-9dffa503b894, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bb4cb419-146e-473c-abad-9dffa503b894,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:34.258233 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:34.262435 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:34.351572 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Pushing event updated for resources: {'Port': ['ID=bb4cb419-146e-473c-abad-9dffa503b894,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:34.355163 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:34.356184 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:34.394404 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Allocated IP 172.20.0.1 (6c217a71-5586-47ee-ab1e-21c28b2d40a1/2b5e1fe5-3648-40fc-a046-12baa71ed42b/af8ad735-24f2-4b00-92d0-5912edb52f10) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:34.417905 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-dc689e51-9f3d-4570-b7a2-8c666d30c312 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:34.515172 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:34.527539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4f232a57-5a8e-4eda-a950-0e15ebdb29f5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:34.563008 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc', '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', '2206a32c-5500-4208-9123-7aab17fc53c7'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:34.572668 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:34.735675 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event deleted for resources: {'Port': ['ID=2206a32c-5500-4208-9123-7aab17fc53c7,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:34.746500 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.219s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:34.938920 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c713f37-8863-42f6-ad83-505b1a815003 req-ccc08a60-5c31-42a1-bcd2-f9d372bf8d8b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.1106830 May 25 16:06:34.957200 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] No DVR routers for this DVR port 2206a32c-5500-4208-9123-7aab17fc53c7 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:34.964469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:34.988510 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'name': 'network-vif-deleted', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:35.033945 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.046466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ff20c3d3-0122-4c2f-998a-5cea2b472eba req-f7362b8a-8d6c-4054-93b3-1ac4a0f5cd2d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:35.070580 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Transition to ACTIVE for port object af8ad735-24f2-4b00-92d0-5912edb52f10 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:35.074449 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ff20c3d3-0122-4c2f-998a-5cea2b472eba req-f7362b8a-8d6c-4054-93b3-1ac4a0f5cd2d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.3752394 May 25 16:06:35.095276 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d834f6a6-1514-4641-a5b3-620eb34e1361 req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "DELETE /v2.0/ports/2206a32c-5500-4208-9123-7aab17fc53c7 HTTP/1.1" status: 204 len: 173 time: 2.0817597 May 25 16:06:35.135842 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9a3c9a0a-5d17-4c73-b579-69f9f807466f', 'name': 'network-vif-deleted', 'tag': '2206a32c-5500-4208-9123-7aab17fc53c7', 'status': 'completed', 'code': 200} May 25 16:06:35.157564 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ff20c3d3-0122-4c2f-998a-5cea2b472eba req-e274e612-3fd8-4328-8803-b59e939d2543 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0820482 May 25 16:06:35.220652 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.234471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7070dbb9-539f-4420-a86d-8ca671aeb033 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:35.291231 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:28Z', 'revision_number': 3, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:34Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:35.299595 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9c64d9e5-c335-48b0-adef-df9fcc6bcd5d None None] The sync data for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi is: [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:28Z', 'revision_number': 3, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:34Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:06:35.395466 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:35.399297 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:35.403499 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:35.403721 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:35.404224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.419453 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Pushing event updated for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:35.423748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.189s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:35.586117 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4297d2c3-f443-4852-b0ae-eeb545d6ccdd req-b92115f1-5304-4162-bd0e-101970225254 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:35.590616 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4297d2c3-f443-4852-b0ae-eeb545d6ccdd req-b92115f1-5304-4162-bd0e-101970225254 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 946 time: 0.2622764 May 25 16:06:35.657663 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4297d2c3-f443-4852-b0ae-eeb545d6ccdd req-3bb0c5ed-0c3d-4551-be8d-c0d4c0857137 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0599203 May 25 16:06:35.740717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.819019 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:35.832764 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:35.832764 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b20655c3-abb2-48c2-8122-ae0d55e085c6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.838392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.903349 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'code': 200} May 25 16:06:35.995319 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:35.997088 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:36.092737 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-163a0584-b7d4-44b4-a076-8013ee18084d None None] Provisioning for port af8ad735-24f2-4b00-92d0-5912edb52f10 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:36.167081 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-163a0584-b7d4-44b4-a076-8013ee18084d None None] Provisioning complete for port af8ad735-24f2-4b00-92d0-5912edb52f10 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:36.167081 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-163a0584-b7d4-44b4-a076-8013ee18084d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:36.307146 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29c8a185-d4e7-4c02-ab7c-58026b705929 req-454ca3ff-0b31-4279-aec0-4a1890322ddb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.0914748 May 25 16:06:36.323443 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=8']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:36.338591 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-163a0584-b7d4-44b4-a076-8013ee18084d None None] Port af8ad735-24f2-4b00-92d0-5912edb52f10 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:06:36.345785 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.346543 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.347065 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:36.347801 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.349131 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.349903 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.351107 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.352096 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.352970 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.353795 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.354591 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.355371 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.356014 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:06:36.365132 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.367s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:36.516751 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:36.518089 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:36.538436 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:36.616007 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-134f4764-9f11-4114-ae6b-dc54da90cf3c req-5b80d3d6-dadd-4a0a-9001-6656c27c8876 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:36.618392 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-134f4764-9f11-4114-ae6b-dc54da90cf3c req-5b80d3d6-dadd-4a0a-9001-6656c27c8876 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1895442 May 25 16:06:36.710725 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-134f4764-9f11-4114-ae6b-dc54da90cf3c req-912da359-daac-4044-8ebd-092d9f578fb7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0823917 May 25 16:06:36.804885 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Pushing event updated for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:36.808223 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.270s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:36.932315 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2c912804-875c-413d-923f-315c2d2959b7 req-134d7aad-ebb3-4303-9c43-1edd1edbb58a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:36.934049 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-2c912804-875c-413d-923f-315c2d2959b7 req-134d7aad-ebb3-4303-9c43-1edd1edbb58a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Synchronizing usage tracker for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:36.976413 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:36.978040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:37.063355 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45264) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:37.066522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-2c912804-875c-413d-923f-315c2d2959b7 req-134d7aad-ebb3-4303-9c43-1edd1edbb58a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Unset dirty status for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:37.068295 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7dd43842-3d27-4de2-ae5e-16c3c35cc194 req-c992b971-2e0f-4051-b4e8-8aea54cc2415 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:37.068519 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2c912804-875c-413d-923f-315c2d2959b7 req-134d7aad-ebb3-4303-9c43-1edd1edbb58a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.6320674 May 25 16:06:37.071798 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7dd43842-3d27-4de2-ae5e-16c3c35cc194 req-c992b971-2e0f-4051-b4e8-8aea54cc2415 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1845784 May 25 16:06:37.132175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7dd43842-3d27-4de2-ae5e-16c3c35cc194 req-eea23cfa-481f-4fcf-b670-e340c7ed5dba tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0529656 May 25 16:06:37.404698 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Deleting subnet 4fa131e2-5aa0-4f62-bf74-2a67a79ad81c {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:06:37.408832 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:37.422395 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2206a32c-5500-4208-9123-7aab17fc53c7 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:37.477829 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 2206a32c-5500-4208-9123-7aab17fc53c7 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:37.478500 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2206a32c-5500-4208-9123-7aab17fc53c7 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:37.479780 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:37.528887 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:37.544752 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:37.660465 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-6a3fd981-385e-496f-8b80-27e79a658c85 None None] Provisioning complete for port af8ad735-24f2-4b00-92d0-5912edb52f10 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:37.679588 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6a3fd981-385e-496f-8b80-27e79a658c85 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:37.705030 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1c70be29-bf9d-4fbf-9ec0-a5460121a554 req-9beac74d-8c56-4fdd-84da-93e29065d4d7 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=9a3c9a0a-5d17-4c73-b579-69f9f807466f HTTP/1.1" status: 200 len: 210 time: 0.1607637 May 25 16:06:37.807550 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bb4cb419-146e-473c-abad-9dffa503b894 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:37.862768 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Router ac04f176-2f32-43dd-b873-426ed50b356d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:06:38.070166 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-6a3fd981-385e-496f-8b80-27e79a658c85 None None] Port af8ad735-24f2-4b00-92d0-5912edb52f10 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:06:38.099612 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afe98a06-0852-4504-8e44-6ee08e94c05d req-2f4f6277-5071-4535-8dd6-36e07e0f96ad tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:38.109141 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afe98a06-0852-4504-8e44-6ee08e94c05d req-2f4f6277-5071-4535-8dd6-36e07e0f96ad tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 945 time: 0.1557276 May 25 16:06:38.183540 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port bb4cb419-146e-473c-abad-9dffa503b894 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:38.188678 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afe98a06-0852-4504-8e44-6ee08e94c05d req-f1dab554-cd8b-4f08-bc51-8c389d716105 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0715246 May 25 16:06:38.190308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bb4cb419-146e-473c-abad-9dffa503b894 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:38.191257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.220323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Deleting port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:38.225706 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.246155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.264196 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:06:38.343569 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.374338 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port bb4cb419-146e-473c-abad-9dffa503b894, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bb4cb419-146e-473c-abad-9dffa503b894,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:38.504132 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3358c764-6b0a-4517-a89b-3f7c66b75b81 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.539652 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.539652 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:38.545564 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:06:38.555873 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] 10.222.0.82 "PUT /v2.0/routers/ac04f176-2f32-43dd-b873-426ed50b356d/add_router_interface HTTP/1.1" status: 200 len: 503 time: 6.7984509 May 25 16:06:38.588155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Sync routers for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:06:38.591500 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] For port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:38.592041 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.623279 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:38.624085 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:38.692997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.830486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:38.832248 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Calling delete_port for a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:38.840029 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d2edcfb1-4b87-4c73-8ba7-370d114af504 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:38.917956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Pushing event deleted for resources: {'Subnet': ['ID=4fa131e2-5aa0-4f62-bf74-2a67a79ad81c,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:38.930741 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:38.948309 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:38.962575 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-2f7db6d5-442a-47c2-87bb-444286cbb59f admin admin] 10.222.0.82 "DELETE /v2.0/subnets/4fa131e2-5aa0-4f62-bf74-2a67a79ad81c HTTP/1.1" status: 204 len: 168 time: 1.8862636 May 25 16:06:38.962780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45354) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:39.123772 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=bb4cb419-146e-473c-abad-9dffa503b894,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:39.142590 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.607s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:39.233190 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.367221 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.373289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9a53192d-5a5a-455c-b59a-76b584284b77 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:39.427085 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.437715 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['76801a33-2412-4d01-aee4-406f117edbd1', 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:39.445045 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['54cf369d-e27b-4d90-a4f4-d729a1eb711e']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:39.484157 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'd91de080-4e40-4f71-aea3-f86a8f72864b') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:39.484611 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'd91de080-4e40-4f71-aea3-f86a8f72864b') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:39.500839 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.501545 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Pushing event deleted for resources: {'Port': ['ID=a7ff877f-0a9b-46af-9cc4-578eeaaa2e89,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:39.510916 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:39.532394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.540425 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Releasing vlan 918 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:06:39.541181 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.620920 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.661045 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bb4cb419-146e-473c-abad-9dffa503b894 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:39.683839 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] No DVR routers for this DVR port a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:39.684753 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.696950 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'name': 'network-vif-deleted', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:06:39.730346 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:06:39.778912 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-2c912804-875c-413d-923f-315c2d2959b7 req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "DELETE /v2.0/ports/a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 HTTP/1.1" status: 204 len: 173 time: 2.6609316 May 25 16:06:39.795510 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2ac1c6ba-5771-41d4-8ba1-93eb640c5da2 req-5681ec25-f7bd-4702-b7ae-145e6fd9827e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 210 time: 0.3594642 May 25 16:06:39.837594 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.839957 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:39.861556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:39.882025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Pushing event deleted for resources: {'Network': ['ID=54cf369d-e27b-4d90-a4f4-d729a1eb711e,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:39.884738 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '43046111-0b2a-499f-8353-b8fd77b989ea', 'name': 'network-vif-deleted', 'tag': 'a7ff877f-0a9b-46af-9cc4-578eeaaa2e89', 'status': 'completed', 'code': 200} May 25 16:06:39.885748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:39.982122 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] 10.222.0.82 "DELETE /v2.0/networks/54cf369d-e27b-4d90-a4f4-d729a1eb711e HTTP/1.1" status: 204 len: 168 time: 1.0263145 May 25 16:06:40.055195 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 2, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:40.064647 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bb4cb419-146e-473c-abad-9dffa503b894 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:40.066571 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.150301 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Checking router: ac04f176-2f32-43dd-b873-426ed50b356d for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:06:40.151092 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Updating router ac04f176-2f32-43dd-b873-426ed50b356d port af8ad735-24f2-4b00-92d0-5912edb52f10 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:06:40.237170 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.240667 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45408) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:40.256369 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1cf1a20-1742-4709-a228-e5d0165f026e req-962f9d36-79e8-49cd-be28-96577ea2b90e tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:40.272046 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1cf1a20-1742-4709-a228-e5d0165f026e req-962f9d36-79e8-49cd-be28-96577ea2b90e tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.1849387 May 25 16:06:40.319714 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Request body: {'network': {'name': 'tempest-TestShelveInstance-864022766-network', 'tenant_id': '649773106ce5438aaa40fb240586ad11'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:40.330994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:40.369991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Usage tracker for resource:network and tenant:649773106ce5438aaa40fb240586ad11 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:40.373781 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:40.380887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:40.391859 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1cf1a20-1742-4709-a228-e5d0165f026e req-d04a4477-63ef-46f7-8295-5bca448861f6 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1185267 May 25 16:06:40.589848 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Made reservation on behalf of 649773106ce5438aaa40fb240586ad11 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:06:40.591353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.612296 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.638494 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.643615 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:40.644921 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Usage tracker for resource:security_group_rule and tenant:649773106ce5438aaa40fb240586ad11 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:40.648708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:40.656357 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:40.716924 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a9ee02c4-3a9d-4d7b-b0bd-02c2aeeed38e req-a933af13-3319-4f13-a4b4-b92b16f69440 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:40.717284 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a9ee02c4-3a9d-4d7b-b0bd-02c2aeeed38e req-a933af13-3319-4f13-a4b4-b92b16f69440 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1708975 May 25 16:06:40.771840 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45428) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:40.855523 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a9ee02c4-3a9d-4d7b-b0bd-02c2aeeed38e req-9a4c48a6-d02b-408f-9635-0567fea90408 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0985596 May 25 16:06:40.925228 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:40.931370 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ec4f71a3-b57a-4199-9a5c-0a7a1168294f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=9e6e907c5841468a85bc59f21b26372d&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1539590 May 25 16:06:40.931851 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45434) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:40.999198 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.019367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.035267 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] vlan segment allocate from pool success with {'vlan_id': 906, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:06:41.058530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.059162 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Added segment 03694269-0677-408f-bbe1-50be69716a73 of type vlan for network a333d7d4-c695-4061-8c6b-d08cd46bd9cf May 25 16:06:41.113373 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Transition to ACTIVE for port object af8ad735-24f2-4b00-92d0-5912edb52f10 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:41.148888 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.181956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.184729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-39d60a2e-b3e3-4a44-bde5-65a84e81cce7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:41.200285 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.229264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.234308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:41.273103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.280013 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-07c1159d-9272-4cb6-90b6-7aa4b7501b49 req-e5ecf71f-3802-4a54-9a5a-df25b235f95b tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=43046111-0b2a-499f-8353-b8fd77b989ea HTTP/1.1" status: 200 len: 210 time: 0.0962188 May 25 16:06:41.305318 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=627fbdb9-a018-40b0-8429-3985a3de0326,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:41.308832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:41.360774 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.385081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1107e7f3-1bc4-43bf-aa22-999bbec9c074 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:41.438321 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7547d244-8c01-4ddb-8286-ed592f9280b0,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:41.469951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:41.479885 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.489817 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/7547d244-8c01-4ddb-8286-ed592f9280b0 HTTP/1.1" status: 204 len: 168 time: 0.5539966 May 25 16:06:41.496573 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.498793 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:41.499472 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:41.509773 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:41.530110 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-155cd717-ccfa-4b0e-bad5-2d094f3c3fa5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:41.610165 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:41.610766 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device a7ff877f-0a9b-46af-9cc4-578eeaaa2e89 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:41.622256 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Pushing event updated for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:41.626636 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.392s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:41.836019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Pushing event updated for resources: {'Network': ['ID=a333d7d4-c695-4061-8c6b-d08cd46bd9cf,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:41.840972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.311s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:41.856577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:06:41.884611 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 881 time: 1.6365883 May 25 16:06:41.887830 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45470) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:41.898451 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Request body: {'subnet': {'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestShelveInstance-864022766-subnet', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:41.923683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:41.936801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f80eb04-2e61-4cba-84e5-56865e5e21ea req-b315cf37-f4fd-400f-b1c8-2f5898b8fe65 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:41.938621 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8f80eb04-2e61-4cba-84e5-56865e5e21ea req-b315cf37-f4fd-400f-b1c8-2f5898b8fe65 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.2124183 May 25 16:06:41.939377 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Usage tracker for resource:subnet and tenant:649773106ce5438aaa40fb240586ad11 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:41.948028 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:41.982416 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:42.009471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-622fe969-db4b-4ec4-8988-e7cf32f32110 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:42.010424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-622fe969-db4b-4ec4-8988-e7cf32f32110 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:42.044635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8f80eb04-2e61-4cba-84e5-56865e5e21ea req-a4223332-9c72-4c16-9692-81ab8fe6396b tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0914717 May 25 16:06:42.097581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Made reservation on behalf of 649773106ce5438aaa40fb240586ad11 for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:06:42.101225 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:42.132549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45484) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:42.169589 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-622fe969-db4b-4ec4-8988-e7cf32f32110 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:42.177080 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-622fe969-db4b-4ec4-8988-e7cf32f32110 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.3930197 May 25 16:06:42.296938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d620c4f8-02bf-4b05-9d71-feb6639d4a3b req-348b8b07-df6e-465b-a2de-98431309603e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:42.300486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Attempting to bind port af8ad735-24f2-4b00-92d0-5912edb52f10 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:42.302053 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Attempting to bind port af8ad735-24f2-4b00-92d0-5912edb52f10 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:42.305845 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d620c4f8-02bf-4b05-9d71-feb6639d4a3b req-348b8b07-df6e-465b-a2de-98431309603e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.2012761 May 25 16:06:42.327761 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Attempting to bind port af8ad735-24f2-4b00-92d0-5912edb52f10 on network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:42.369273 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:42.378862 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d620c4f8-02bf-4b05-9d71-feb6639d4a3b req-1affbe3c-98ce-441c-bc28-e1941407f0f8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0716763 May 25 16:06:42.381865 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Checking segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:42.382707 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:42.384562 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Bound using segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:42.385445 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Bound port: af8ad735-24f2-4b00-92d0-5912edb52f10, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:42.387977 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:42.585309 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] For port af8ad735-24f2-4b00-92d0-5912edb52f10, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:42.590406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] For port af8ad735-24f2-4b00-92d0-5912edb52f10, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=af8ad735-24f2-4b00-92d0-5912edb52f10,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:42.654906 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Deleting port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:42.655577 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:42.686915 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-aee9ab32-3b94-47d7-ad6a-9a651fb18726 service neutron] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.5258579 May 25 16:06:42.696646 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:42.706620 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:42.835805 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Pushing event updated for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:42.844800 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:42.871143 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:42.872687 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:43.075041 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] For port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:43.076048 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:43.300457 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:43.415422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Calling delete_port for 5c5712c5-651e-4b21-a6fc-b32aff76f3cb owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:43.719995 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f494c1ed-c7b5-488d-9506-4dd95027744d req-869775bd-b428-417f-a75b-e442e0d902cc tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:43.724876 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f494c1ed-c7b5-488d-9506-4dd95027744d req-869775bd-b428-417f-a75b-e442e0d902cc tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 945 time: 0.2827432 May 25 16:06:43.758251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:43.767320 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:43.773002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-86695ae8-d56e-4047-93eb-75c456e808d3 req-02fdedcc-5ce0-4d3e-a878-d0eb7caa9d1f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:43.774726 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86695ae8-d56e-4047-93eb-75c456e808d3 req-02fdedcc-5ce0-4d3e-a878-d0eb7caa9d1f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2258322 May 25 16:06:43.821668 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['3935ec55-2a13-43b8-b0bf-f9ef947051dc'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:06:43.827998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event deleted for resources: {'Port': ['ID=5c5712c5-651e-4b21-a6fc-b32aff76f3cb,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:43.839036 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:43.839576 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f494c1ed-c7b5-488d-9506-4dd95027744d req-c234633e-14d6-4800-ab96-37090b124d30 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1035531 May 25 16:06:43.873353 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-86695ae8-d56e-4047-93eb-75c456e808d3 req-eb585c07-1c21-49dd-8134-a36c1f3be35f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0836205 May 25 16:06:43.958998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] No DVR routers for this DVR port 5c5712c5-651e-4b21-a6fc-b32aff76f3cb on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:43.966438 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:43.996824 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'name': 'network-vif-deleted', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:44.086400 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:42Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:44.126127 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-21ac9a80-ed73-41af-aeb5-5c57515fbcb1 None None] The sync data for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi is: [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:42Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:06:44.149019 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '296a79a0-3c84-4514-b7ce-da1c961013d7', 'name': 'network-vif-deleted', 'tag': '5c5712c5-651e-4b21-a6fc-b32aff76f3cb', 'status': 'completed', 'code': 200} May 25 16:06:44.225281 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-3318c1c8-22e9-46cf-a981-40226bb7b1bc req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "DELETE /v2.0/ports/5c5712c5-651e-4b21-a6fc-b32aff76f3cb HTTP/1.1" status: 204 len: 173 time: 2.0467765 May 25 16:06:44.318445 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:44.328223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d374c1ca-7af2-4c3c-9347-101c5f635095 req-9d85f445-db67-4766-b3e7-749e69db25d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:44.329507 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e617cd8f-10a8-4f6f-9303-8be387b4b680 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:44.329972 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d374c1ca-7af2-4c3c-9347-101c5f635095 req-9d85f445-db67-4766-b3e7-749e69db25d2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.3045752 May 25 16:06:44.439053 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d374c1ca-7af2-4c3c-9347-101c5f635095 req-217732f9-965e-454a-b8d0-55efe9384641 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0942242 May 25 16:06:44.503471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Pushing event updated for resources: {'Subnet': ['ID=00016ef9-d427-4aab-906b-515d38fbeb52,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:44.514047 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:44.523331 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Checking segment: {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:44.524505 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Network hostable DHCP agents. Network: a333d7d4-c695-4061-8c6b-d08cd46bd9cf, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:06:44.704066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:06:44.874684 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, a333d7d4-c695-4061-8c6b-d08cd46bd9cf) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:44.875598 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c4a57d79-d4cc-44aa-9661-72401b579b93 None None] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:06:44.882517 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:44.883434 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.058096 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 844 time: 3.1681478 May 25 16:06:45.060811 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45592) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:45.067945 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Request body: {'router': {'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:45.082849 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:45.096392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Usage tracker for resource:router and tenant:649773106ce5438aaa40fb240586ad11 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:45.098971 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:45.117870 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:45.176482 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9ce24b1c-7fe6-47a6-800b-e14d4168f2a0 req-e8ef05d5-bae6-4bda-9e62-59b3268c4fbd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=296a79a0-3c84-4514-b7ce-da1c961013d7 HTTP/1.1" status: 200 len: 210 time: 0.0802057 May 25 16:06:45.239773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.249658 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.255796 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45596) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:45.264574 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2b0e214a-f66a-4043-a2cb-dd8222a42f8b req-8cc68452-addb-4a96-8343-2fbc10b62db4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:45.273829 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2b0e214a-f66a-4043-a2cb-dd8222a42f8b req-8cc68452-addb-4a96-8343-2fbc10b62db4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.2435749 May 25 16:06:45.307760 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.361379 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2b0e214a-f66a-4043-a2cb-dd8222a42f8b req-adf42e61-59d9-47e6-a58f-8b79930d2196 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0842149 May 25 16:06:45.407003 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'fixed_ips': [{'subnet_id': '00016ef9-d427-4aab-906b-515d38fbeb52'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:06:45.407845 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.449562 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5c5712c5-651e-4b21-a6fc-b32aff76f3cb no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:45.537467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 5c5712c5-651e-4b21-a6fc-b32aff76f3cb {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:45.537826 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5c5712c5-651e-4b21-a6fc-b32aff76f3cb not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:45.624535 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device af8ad735-24f2-4b00-92d0-5912edb52f10 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:45.756060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0dddf36-2f81-4a2f-bf5b-2b530ef02d30 req-def67b70-2877-4e74-9e77-81ec0868e4cb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:45.758183 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0dddf36-2f81-4a2f-bf5b-2b530ef02d30 req-def67b70-2877-4e74-9e77-81ec0868e4cb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1474071 May 25 16:06:45.774705 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Allocated IP 172.20.0.2 (a333d7d4-c695-4061-8c6b-d08cd46bd9cf/00016ef9-d427-4aab-906b-515d38fbeb52/ba27d822-ea42-4755-a4f4-b477a76c022b) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:45.861915 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0dddf36-2f81-4a2f-bf5b-2b530ef02d30 req-6db80347-1c8a-4968-86ee-a5601d9c7ec7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0937068 May 25 16:06:45.885568 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:45.912795 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d352564d-b41f-44c6-a4f4-8ba74749c8cb admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=625c6d6c1b4949ce817bdff51006604f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6552343 May 25 16:06:45.915623 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port af8ad735-24f2-4b00-92d0-5912edb52f10 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:45.921168 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45606) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:45.935616 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port af8ad735-24f2-4b00-92d0-5912edb52f10 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:45.936107 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.009305 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.055318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.165022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.182722 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Transition to ACTIVE for port object ba27d822-ea42-4755-a4f4-b477a76c022b will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:46.185285 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.227915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Transition to ACTIVE for port object ba27d822-ea42-4755-a4f4-b477a76c022b will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:46.236952 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port af8ad735-24f2-4b00-92d0-5912edb52f10, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=af8ad735-24f2-4b00-92d0-5912edb52f10,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:46.346454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.351124 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.360155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:46.378359 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.394233 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b904c93c-2c8a-44f6-960b-292a5d391022 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:46.412263 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=71d2df97-673b-4fb7-bedf-e2c7419ef5a7,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:46.501344 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:46.507585 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:46.514918 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:46.570350 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:46.571615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.596665 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:46.612752 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.695148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-abc5a11b-8727-4b3e-b17e-65f692dfc3a5 req-90992950-5467-4c99-af9f-93e456615b5f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:46.702926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-abc5a11b-8727-4b3e-b17e-65f692dfc3a5 req-90992950-5467-4c99-af9f-93e456615b5f tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1830938 May 25 16:06:46.806138 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-abc5a11b-8727-4b3e-b17e-65f692dfc3a5 req-fecf7b41-8a9b-4934-8e1b-94bcc57f6f1c tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1020041 May 25 16:06:46.821129 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Attempting to bind port ba27d822-ea42-4755-a4f4-b477a76c022b on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:46.821928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Attempting to bind port ba27d822-ea42-4755-a4f4-b477a76c022b by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:46.822647 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Attempting to bind port ba27d822-ea42-4755-a4f4-b477a76c022b on network a333d7d4-c695-4061-8c6b-d08cd46bd9cf {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:46.835002 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Pushing event updated for resources: {'Port': ['ID=ba27d822-ea42-4755-a4f4-b477a76c022b,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:46.838190 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:46.838807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Checking segment: {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:46.839406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.839896 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Bound using segment: {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:46.840504 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Bound port: ba27d822-ea42-4755-a4f4-b477a76c022b, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:46.840986 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.853404 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.494s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:46.855328 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.430s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:46.855328 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.001s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:46.895303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:46.900486 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/71d2df97-673b-4fb7-bedf-e2c7419ef5a7 HTTP/1.1" status: 204 len: 168 time: 0.9777799 May 25 16:06:46.997729 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] For port ba27d822-ea42-4755-a4f4-b477a76c022b, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:47.002746 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] For port ba27d822-ea42-4755-a4f4-b477a76c022b, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=ba27d822-ea42-4755-a4f4-b477a76c022b,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:47.006076 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Allocated IP 172.27.2.36 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/111d98cb-22ae-448d-b6d3-127948e3a74f) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:47.142410 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:47.152436 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:47.184322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e82ec64-71b6-4a53-99e9-c4043a666e2d req-21e69af5-3b7f-4552-b7cc-06e9ed48398a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:47.187800 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e82ec64-71b6-4a53-99e9-c4043a666e2d req-21e69af5-3b7f-4552-b7cc-06e9ed48398a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1152203 May 25 16:06:47.229703 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:47.239610 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Pushing event updated for resources: {'Port': ['ID=ba27d822-ea42-4755-a4f4-b477a76c022b,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:47.252310 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:47.290825 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e82ec64-71b6-4a53-99e9-c4043a666e2d req-d20d1671-0a77-490d-8e45-f51e2e337682 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0849068 May 25 16:06:47.420917 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:47.434829 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bcfd2e05-9e94-4b2f-a4e0-e97e1df15998 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:47.567733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:47.573248 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:47.577982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:47.636818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device af8ad735-24f2-4b00-92d0-5912edb52f10 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:47.758150 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port af8ad735-24f2-4b00-92d0-5912edb52f10 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:47.758691 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:47.851899 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Pushing event updated for resources: {'Port': ['ID=111d98cb-22ae-448d-b6d3-127948e3a74f,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:47.856866 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.422s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:48.135313 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5c8cd69-fc23-4a54-a496-5506ffacff8d req-6f369bad-0a9b-4160-ba54-81070ac8ea4d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:48.137299 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5c8cd69-fc23-4a54-a496-5506ffacff8d req-6f369bad-0a9b-4160-ba54-81070ac8ea4d tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1303954 May 25 16:06:48.265539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:48.267673 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5c8cd69-fc23-4a54-a496-5506ffacff8d req-2ef8e20a-0839-4ffe-b3f8-a616338b417a tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1172380 May 25 16:06:48.448001 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-7eff3298-4063-484f-a185-5f3cf8d28d01 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:48.450169 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-7eff3298-4063-484f-a185-5f3cf8d28d01 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 1097 time: 0.2949982 May 25 16:06:48.552322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6127baa6-e877-498c-809c-d6763c594c07 req-859d3e02-24be-4174-8ffc-b04aa22b9304 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:48.559735 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6127baa6-e877-498c-809c-d6763c594c07 req-859d3e02-24be-4174-8ffc-b04aa22b9304 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1106436 May 25 16:06:48.578336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-2191b1e9-63f7-4027-98ff-4bf5e6542a8f None None] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:06:48.661963 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6127baa6-e877-498c-809c-d6763c594c07 req-0961a87f-b44d-4b82-add9-64167626832c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0970221 May 25 16:06:48.761229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-3a7cf523-553b-40db-b0f3-6e77b9d1492d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:48.765895 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-3a7cf523-553b-40db-b0f3-6e77b9d1492d service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1097 time: 0.2205312 May 25 16:06:48.768276 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:48.844411 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-2c115640-f413-4723-815f-80fb48104db1 None None] Provisioning for port ba27d822-ea42-4755-a4f4-b477a76c022b completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:49.128003 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3202b47f-6fa1-4596-9ab3-5dd0ec1cb96d req-ef25633f-9c46-41b4-9dbb-043a03098f3a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.129772 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3202b47f-6fa1-4596-9ab3-5dd0ec1cb96d req-ef25633f-9c46-41b4-9dbb-043a03098f3a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.1346607 May 25 16:06:49.194851 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3202b47f-6fa1-4596-9ab3-5dd0ec1cb96d req-a6ccaf44-d0fe-4635-8466-6fbe8c48eecb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0573194 May 25 16:06:49.278362 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-a8632871-7ac1-425d-be3a-24a716199943 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.280802 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-a8632871-7ac1-425d-be3a-24a716199943 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.5076535 May 25 16:06:49.416614 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-8caad6f9-e348-4c52-8367-949f604559bb service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1000 time: 0.1278763 May 25 16:06:49.515313 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-7c78a809-eae2-4571-8c96-a8c6a85bb664 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.523402 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-7c78a809-eae2-4571-8c96-a8c6a85bb664 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0936356 May 25 16:06:49.545862 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-36570129-ee5e-4d8b-b8f1-0343f1f5efe0 req-2421a66c-b6fb-4d84-89cb-0300245727f0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.547437 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-36570129-ee5e-4d8b-b8f1-0343f1f5efe0 req-2421a66c-b6fb-4d84-89cb-0300245727f0 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.1407411 May 25 16:06:49.616560 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-36570129-ee5e-4d8b-b8f1-0343f1f5efe0 req-b93de8a9-86d9-4c93-8257-939772b13596 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0567458 May 25 16:06:49.628156 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Router 2fc03949-9293-45c1-882e-c9127b336a3c is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99544) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:06:49.638458 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device ba27d822-ea42-4755-a4f4-b477a76c022b up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:49.715441 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-96178acf-6a75-4cc6-a2de-d35543d89124 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.716987 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-96178acf-6a75-4cc6-a2de-d35543d89124 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1714723 May 25 16:06:49.788559 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port ba27d822-ea42-4755-a4f4-b477a76c022b completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:49.794425 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port ba27d822-ea42-4755-a4f4-b477a76c022b triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:49.794961 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:49.915751 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b947ce7-ccae-424b-90d3-73628bd4f0d1 req-b8992125-f855-45fa-bcef-537cb25078c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:49.923522 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3b947ce7-ccae-424b-90d3-73628bd4f0d1 req-b8992125-f855-45fa-bcef-537cb25078c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.0995145 May 25 16:06:49.954623 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:06:49.971194 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Sync routers for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:06:49.992054 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:49.992690 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:50.000928 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3b947ce7-ccae-424b-90d3-73628bd4f0d1 req-2e8d368c-e012-4f81-ae06-e6c94223aba2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0723917 May 25 16:06:50.001280 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:50.004298 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-f17dae8b-848f-49ff-8711-facd5dc911fd service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2647569 May 25 16:06:50.038766 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port ba27d822-ea42-4755-a4f4-b477a76c022b, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=ba27d822-ea42-4755-a4f4-b477a76c022b,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:50.154334 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:50.157877 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:50.178844 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:50.184681 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-0479254b-b345-4935-9546-eebfa0ceac1e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:50.188961 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-0479254b-b345-4935-9546-eebfa0ceac1e tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.0980604 May 25 16:06:50.189106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:50.190207 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 919 time: 5.1281559 May 25 16:06:50.195174 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45678) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:50.274989 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=ba27d822-ea42-4755-a4f4-b477a76c022b,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:50.280088 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.122s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:50.315066 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-f55439cd-887a-412e-946a-f2e8b4c32c48 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2263844 May 25 16:06:50.551425 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:47Z', 'revision_number': 3, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:47Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:50.556216 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Checking router: 2fc03949-9293-45c1-882e-c9127b336a3c for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:06:50.556862 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Updating router 2fc03949-9293-45c1-882e-c9127b336a3c port 111d98cb-22ae-448d-b6d3-127948e3a74f binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:06:50.614856 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:50.813464 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ada972c3-e9c2-4944-be1e-02fd4ae1fed6 req-252ea9e4-1a03-4a80-b761-61d5f36f9ec7 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:50.815039 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ada972c3-e9c2-4944-be1e-02fd4ae1fed6 req-252ea9e4-1a03-4a80-b761-61d5f36f9ec7 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.4241705 May 25 16:06:51.010455 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ada972c3-e9c2-4944-be1e-02fd4ae1fed6 req-7e3c10f9-541b-4e20-b297-6beedb23a712 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1786854 May 25 16:06:51.015062 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Deleting port 76801a33-2412-4d01-aee4-406f117edbd1 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:06:51.015160 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.117945 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.148013 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Transition to ACTIVE for port object 111d98cb-22ae-448d-b6d3-127948e3a74f will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:51.257813 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.264680 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f08883da-e0e2-4564-a24b-cf6302678b45 req-d63aaffd-952c-4a7e-b739-ce05376ce0b7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:51.266882 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-f55db251-f4a0-47fb-91d8-e174f1b9b330 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:51.270812 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f08883da-e0e2-4564-a24b-cf6302678b45 req-d63aaffd-952c-4a7e-b739-ce05376ce0b7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 945 time: 0.5144410 May 25 16:06:51.352703 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Attempting to bind port 111d98cb-22ae-448d-b6d3-127948e3a74f on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:51.353055 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Attempting to bind port 111d98cb-22ae-448d-b6d3-127948e3a74f by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:51.353466 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Attempting to bind port 111d98cb-22ae-448d-b6d3-127948e3a74f on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:51.359905 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cefd42ec-e483-4cda-a46d-aa99ccac0d64 None None] Pushing event updated for resources: {'Port': ['ID=111d98cb-22ae-448d-b6d3-127948e3a74f,revision_number=2']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:51.365154 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-cefd42ec-e483-4cda-a46d-aa99ccac0d64 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:51.366170 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:51.366550 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:51.367055 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.367560 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:51.368233 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Bound port: 111d98cb-22ae-448d-b6d3-127948e3a74f, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:51.368637 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.501423 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] For port 76801a33-2412-4d01-aee4-406f117edbd1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=76801a33-2412-4d01-aee4-406f117edbd1,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:51.501786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.531087 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f08883da-e0e2-4564-a24b-cf6302678b45 req-c1d2850d-55c4-4e74-9e5a-6b8c30936bb6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/security-groups?id=c255724c-5dbd-46d2-bb27-077184561f8d&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2500215 May 25 16:06:51.547197 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] For port 111d98cb-22ae-448d-b6d3-127948e3a74f, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:51.551382 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] For port 111d98cb-22ae-448d-b6d3-127948e3a74f, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=111d98cb-22ae-448d-b6d3-127948e3a74f,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:51.635434 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.640149 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:51.641196 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device ba27d822-ea42-4755-a4f4-b477a76c022b up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:51.702702 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-78937080-7bad-496b-aaef-38f5dfc98081 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:51.704261 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-78937080-7bad-496b-aaef-38f5dfc98081 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Synchronizing usage tracker for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:51.804155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port ba27d822-ea42-4755-a4f4-b477a76c022b triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:51.804795 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:51.826394 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:51.827487 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:51.830729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Calling delete_port for 76801a33-2412-4d01-aee4-406f117edbd1 owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:06:51.861048 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-78937080-7bad-496b-aaef-38f5dfc98081 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Unset dirty status for tenant:ed476c44700b47648958b0d2e020320d on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:51.862208 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-78937080-7bad-496b-aaef-38f5dfc98081 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.4317937 May 25 16:06:51.950783 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cefd42ec-e483-4cda-a46d-aa99ccac0d64 None None] Pushing event updated for resources: {'Port': ['ID=111d98cb-22ae-448d-b6d3-127948e3a74f,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:51.981367 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-cefd42ec-e483-4cda-a46d-aa99ccac0d64 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.341s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:52.220955 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:52.485102 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba326843-a097-4491-a6bb-91b4d7a96658 req-8e45b1a3-1cc8-4fdf-85a8-38ac819f1d6c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:52.487323 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ba326843-a097-4491-a6bb-91b4d7a96658 req-8e45b1a3-1cc8-4fdf-85a8-38ac819f1d6c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 1.2150202 May 25 16:06:52.637902 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ba326843-a097-4491-a6bb-91b4d7a96658 req-d6a7831a-2a65-4506-a9a1-6d021c6e3f1f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1393836 May 25 16:06:52.753310 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ec0fbf1c-79a0-4584-adfc-fcd881b664e2 req-dcf119b1-dd4b-49bf-bb8e-729cb3fd51e0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:52.762610 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ec0fbf1c-79a0-4584-adfc-fcd881b664e2 req-dcf119b1-dd4b-49bf-bb8e-729cb3fd51e0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.4135330 May 25 16:06:52.781966 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:47Z', 'revision_number': 3, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:51Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:52.796918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:52.806220 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-09d5fa17-3a25-4bbb-9b28-bf11c7774ee1 None None] The sync data for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi is: [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:47Z', 'revision_number': 3, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:51Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:06:52.940816 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ec0fbf1c-79a0-4584-adfc-fcd881b664e2 req-b1bddba4-6ebb-408a-b22d-468515da5a02 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1742721 May 25 16:06:52.948109 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:52.967789 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2df48033-30bd-412b-829f-13035368b725 req-e76e97dd-6e09-42d7-9dfd-5df00e755204 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0677536 May 25 16:06:52.975231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:53.046245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Pushing event deleted for resources: {'Port': ['ID=76801a33-2412-4d01-aee4-406f117edbd1,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:53.048884 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:53.212996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] No DVR routers for this DVR port 76801a33-2412-4d01-aee4-406f117edbd1 on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:06:53.228353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:53.252922 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'name': 'network-vif-deleted', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:53.304508 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3e118469-fc2f-4397-9849-a89bcf9f0173 req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.42 "DELETE /v2.0/ports/76801a33-2412-4d01-aee4-406f117edbd1 HTTP/1.1" status: 204 len: 173 time: 3.1085389 May 25 16:06:53.468433 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'd5ad0bd5-f89c-49d8-bd24-4208d957fb2a', 'name': 'network-vif-deleted', 'tag': '76801a33-2412-4d01-aee4-406f117edbd1', 'status': 'completed', 'code': 200} May 25 16:06:53.558837 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 76801a33-2412-4d01-aee4-406f117edbd1 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:53.593113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Allocated IP 172.20.0.1 (a333d7d4-c695-4061-8c6b-d08cd46bd9cf/00016ef9-d427-4aab-906b-515d38fbeb52/d69a72d5-419a-489c-b21f-eee55561f332) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:06:53.716607 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 76801a33-2412-4d01-aee4-406f117edbd1 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:06:53.731450 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 76801a33-2412-4d01-aee4-406f117edbd1 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:06:53.734275 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 3935ec55-2a13-43b8-b0bf-f9ef947051dc no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:06:53.827175 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 111d98cb-22ae-448d-b6d3-127948e3a74f up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:54.149898 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.231404 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:06:54.251636 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:54.258374 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5282d6c-bf84-4caf-b36c-ec90d3a86b55 req-f17d1565-195d-4bae-b83b-6f29598461e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:54.260468 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5282d6c-bf84-4caf-b36c-ec90d3a86b55 req-f17d1565-195d-4bae-b83b-6f29598461e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.3368070 May 25 16:06:54.266212 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.270703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 111d98cb-22ae-448d-b6d3-127948e3a74f completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:54.295972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 111d98cb-22ae-448d-b6d3-127948e3a74f triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:54.295972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.310673 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Transition to ACTIVE for port object d69a72d5-419a-489c-b21f-eee55561f332 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:54.348427 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5282d6c-bf84-4caf-b36c-ec90d3a86b55 req-a019b80a-80ab-48f9-a712-11d6d33bac11 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0749254 May 25 16:06:54.481251 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.509347 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 111d98cb-22ae-448d-b6d3-127948e3a74f, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=111d98cb-22ae-448d-b6d3-127948e3a74f,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:06:54.597322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3107074-b59b-4d6c-b36f-8e4cd2f4d905 req-3d659f3b-3d9b-4729-8696-cd43339a56bf tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:54.599117 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3107074-b59b-4d6c-b36f-8e4cd2f4d905 req-3d659f3b-3d9b-4729-8696-cd43339a56bf tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 946 time: 0.0982578 May 25 16:06:54.650897 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3107074-b59b-4d6c-b36f-8e4cd2f4d905 req-ce078f8f-807e-4e17-a410-4a75365faa99 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0451245 May 25 16:06:54.821765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-060d3993-0ea3-4aa1-9186-a9ff8a7f1ff9 req-47990976-6b34-4dac-b1e5-edf181f411b6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Synchronizing usage tracker for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:06:54.841403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.846355 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:54.890425 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.911105 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:54.915515 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:54.969742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=6']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:54.974451 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.128s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:54.975234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.054s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:55.056995 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=111d98cb-22ae-448d-b6d3-127948e3a74f,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:55.068198 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Pushing event updated for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:55.072706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:55.073591 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-060d3993-0ea3-4aa1-9186-a9ff8a7f1ff9 req-47990976-6b34-4dac-b1e5-edf181f411b6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Unset dirty status for tenant:d20a1b70ac3f4086aac2870ee8c08efe on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:06:55.078336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:55.087305 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-060d3993-0ea3-4aa1-9186-a9ff8a7f1ff9 req-47990976-6b34-4dac-b1e5-edf181f411b6 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.3255947 May 25 16:06:55.091439 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:55.096745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:55.110854 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:06:55.204556 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'code': 200} May 25 16:06:55.248451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:55.548406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a3e84b77-767d-43c8-b979-45bc43066f08 None None] Provisioning for port d69a72d5-419a-489c-b21f-eee55561f332 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:06:55.557323 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a3e84b77-767d-43c8-b979-45bc43066f08 None None] Provisioning complete for port d69a72d5-419a-489c-b21f-eee55561f332 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:55.557939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a3e84b77-767d-43c8-b979-45bc43066f08 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:55.597897 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:55.606856 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 111d98cb-22ae-448d-b6d3-127948e3a74f up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:06:55.733958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1135d999-68e1-4c44-b39a-31fd08ef6644 req-77d84ea0-35fd-4e7f-ad21-b033d0969260 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:55.737437 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1135d999-68e1-4c44-b39a-31fd08ef6644 req-77d84ea0-35fd-4e7f-ad21-b033d0969260 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.2070665 May 25 16:06:55.796830 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-a3e84b77-767d-43c8-b979-45bc43066f08 None None] Port d69a72d5-419a-489c-b21f-eee55561f332 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:06:55.830915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 111d98cb-22ae-448d-b6d3-127948e3a74f triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:55.830915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:56.026609 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1135d999-68e1-4c44-b39a-31fd08ef6644 req-133d7805-e687-4632-86cc-b3eb3ea002f5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.2286897 May 25 16:06:56.157656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:56.223406 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:56.259434 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:56.404583 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Pushing event updated for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:56.408558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:56.413270 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:56.539557 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4862a2f9-a408-4e13-a605-bf48ee287271 req-8db35881-6457-4757-9b49-0c3fbf7c4ea6 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:56.541139 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4862a2f9-a408-4e13-a605-bf48ee287271 req-8db35881-6457-4757-9b49-0c3fbf7c4ea6 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.2828755 May 25 16:06:56.629067 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:06:56.632189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:06:56.641020 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4862a2f9-a408-4e13-a605-bf48ee287271 req-22041f2a-3155-4f0f-8928-aa6923f18367 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0783522 May 25 16:06:56.924425 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a2a18301-3d1d-4e5d-b443-e13d57300bdd req-9d7143ee-d3ea-408c-beb0-9a5356a60f54 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0715404 May 25 16:06:56.940231 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-7abec6b1-42dc-4a71-bef9-aeeb9a77d5ff None None] Provisioning complete for port d69a72d5-419a-489c-b21f-eee55561f332 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:06:56.940867 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7abec6b1-42dc-4a71-bef9-aeeb9a77d5ff None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:56.975322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Router 2fc03949-9293-45c1-882e-c9127b336a3c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:06:57.020236 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-7abec6b1-42dc-4a71-bef9-aeeb9a77d5ff None None] Port d69a72d5-419a-489c-b21f-eee55561f332 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:06:57.317103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:06:57.324856 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] 10.222.0.82 "PUT /v2.0/routers/2fc03949-9293-45c1-882e-c9127b336a3c/add_router_interface HTTP/1.1" status: 200 len: 503 time: 7.1282063 May 25 16:06:57.339408 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Sync routers for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:06:57.356328 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a651618f-19ad-455d-9815-201f071e02fa req-0ad14ae0-88e3-4195-863d-3721674afe66 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:57.356652 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:06:57.357647 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:57.378144 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a651618f-19ad-455d-9815-201f071e02fa req-0ad14ae0-88e3-4195-863d-3721674afe66 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1696126 May 25 16:06:57.481817 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a651618f-19ad-455d-9815-201f071e02fa req-4528e4ea-b7f9-4224-a56b-44077358a621 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1014731 May 25 16:06:57.653295 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:06:58.166231 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:55Z', 'revision_number': 4, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:54Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'd69a72d5-419a-489c-b21f-eee55561f332', 'name': '', 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'mac_address': 'fa:16:3e:c7:18:23', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_interface', 'standard_attr_id': 364, 'fixed_ips': [{'subnet_id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:52Z', 'updated_at': '2021-05-25T16:06:55Z', 'revision_number': 2, 'project_id': '649773106ce5438aaa40fb240586ad11', 'subnets': [{'id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:06:58.196087 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Checking router: 2fc03949-9293-45c1-882e-c9127b336a3c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:06:58.196666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Updating router 2fc03949-9293-45c1-882e-c9127b336a3c port d69a72d5-419a-489c-b21f-eee55561f332 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:06:58.272169 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:58.720279 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45770) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:58.729836 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-1065377766', 'description': 'tempest-secgroup-smoke-1065377766 description', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:58.752721 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:58.769445 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:58.779249 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:58.795587 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:58.864068 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:58.889703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Transition to ACTIVE for port object d69a72d5-419a-489c-b21f-eee55561f332 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:06:58.898061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:58.934477 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:58.939918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:58.945429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:58.954717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:58.976993 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:58.989784 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:59.095315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32dad71d-be58-48ec-b0f7-350c0d5c1c3c req-cf04da33-71e8-4518-83e0-6f4dfd616d29 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:59.097365 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32dad71d-be58-48ec-b0f7-350c0d5c1c3c req-cf04da33-71e8-4518-83e0-6f4dfd616d29 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.1393766 May 25 16:06:59.147884 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.206373 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32dad71d-be58-48ec-b0f7-350c0d5c1c3c req-09a0bb44-b7ea-499a-a18d-6c2ce884979a tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0952775 May 25 16:06:59.247919 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Pushing event updated for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:59.250238 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.254772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-353f575c-5fa5-453c-a0e8-b093537d9699 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:06:59.278778 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.297s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:59.335092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroup': ['ID=503c75ec-7911-4bdd-93c2-bfebd1e51c71,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:06:59.347386 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:06:59.392704 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Attempting to bind port d69a72d5-419a-489c-b21f-eee55561f332 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:06:59.406399 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Attempting to bind port d69a72d5-419a-489c-b21f-eee55561f332 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:06:59.406847 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Attempting to bind port d69a72d5-419a-489c-b21f-eee55561f332 on network a333d7d4-c695-4061-8c6b-d08cd46bd9cf {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:06:59.418993 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 6, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:06:59.421657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Checking segment: {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:06:59.421657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.421657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Bound using segment: {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:06:59.421657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Bound port: d69a72d5-419a-489c-b21f-eee55561f332, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '03694269-0677-408f-bbe1-50be69716a73', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 906, 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:06:59.421657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.422322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.427417 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45780) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:06:59.433384 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:06:59.437844 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.7154703 May 25 16:06:59.444173 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:06:59.453969 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:06:59.465865 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d0821c-56a5-4d88-a337-d09dd072ccfa req-a47cb92e-85b2-4610-8864-e8c9ce9d07d7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0854964 May 25 16:06:59.473073 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:06:59.493837 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:06:59.609272 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.688782 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] For port d69a72d5-419a-489c-b21f-eee55561f332, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:06:59.693723 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] For port d69a72d5-419a-489c-b21f-eee55561f332, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d69a72d5-419a-489c-b21f-eee55561f332,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:06:59.730895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:06:59.824435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9c653a24-771c-441e-9c9b-06c4e2e0f87b req-d12388d4-ddec-49c5-b9de-5dde17a175c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:06:59.825977 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9c653a24-771c-441e-9c9b-06c4e2e0f87b req-d12388d4-ddec-49c5-b9de-5dde17a175c2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1443753 May 25 16:06:59.887951 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9c653a24-771c-441e-9c9b-06c4e2e0f87b req-4d24fcce-1a41-44d4-85a4-1082788e2295 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0522299 May 25 16:07:00.137997 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.142754 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:00.166277 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.170096 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:00.206005 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=0f8b6464-ec4c-4b8c-9b4d-d93ec90ddeb3,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:00.210567 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:00.254432 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Pushing event updated for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:00.259918 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:00.262801 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:07:00.264971 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:06:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:00.297406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.306476 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45786) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:00.306641 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.8743374 May 25 16:07:00.315558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:00.332684 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:00.341214 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:00.345500 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:00.376156 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:00.518435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.560223 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:07:00.589683 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.616929 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39a771fa-02dc-496f-9b21-df7cae89f960 req-aa6d60ac-2c71-4659-a14b-768e7d64fca0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:00.618765 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-39a771fa-02dc-496f-9b21-df7cae89f960 req-aa6d60ac-2c71-4659-a14b-768e7d64fca0 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.1997182 May 25 16:07:00.691519 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-39a771fa-02dc-496f-9b21-df7cae89f960 req-51557b82-ad62-4936-8774-9a80aeaa7271 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0642102 May 25 16:07:00.714845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:00.726369 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:00.775009 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=cadd288b-15c1-4cca-9292-328a83e1f5e2,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:00.782023 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:01.073259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.075214 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.7709484 May 25 16:07:01.078372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45796) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:01.085032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:01.089706 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d6e5f5a-9cb5-4b98-aa73-4f32c595824e req-78381894-b7b3-4cf3-96fe-847971a656f8 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.2599187 May 25 16:07:01.093828 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:01.108373 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:01.116476 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:01.131264 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:01.142941 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:55Z', 'revision_number': 4, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:54Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'd69a72d5-419a-489c-b21f-eee55561f332', 'name': '', 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'mac_address': 'fa:16:3e:c7:18:23', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_interface', 'standard_attr_id': 364, 'fixed_ips': [{'subnet_id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:52Z', 'updated_at': '2021-05-25T16:06:59Z', 'revision_number': 4, 'project_id': '649773106ce5438aaa40fb240586ad11', 'subnets': [{'id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:01.148360 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b558a80a-b7f7-44e8-aa97-06477a13f01d None None] The sync data for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi is: [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:06:55Z', 'revision_number': 4, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:54Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'd69a72d5-419a-489c-b21f-eee55561f332', 'name': '', 'network_id': 'a333d7d4-c695-4061-8c6b-d08cd46bd9cf', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'mac_address': 'fa:16:3e:c7:18:23', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_interface', 'standard_attr_id': 364, 'fixed_ips': [{'subnet_id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:52Z', 'updated_at': '2021-05-25T16:06:59Z', 'revision_number': 4, 'project_id': '649773106ce5438aaa40fb240586ad11', 'subnets': [{'id': '00016ef9-d427-4aab-906b-515d38fbeb52', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:07:01.287885 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.386520 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.437607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bafbec43-79f5-49b7-872f-51e72bc49561 req-10d8c139-a7e1-49cc-b752-fcbbcce94065 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:01.445883 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bafbec43-79f5-49b7-872f-51e72bc49561 req-10d8c139-a7e1-49cc-b752-fcbbcce94065 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1674385 May 25 16:07:01.478542 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.486491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:01.511718 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d4c0ea38-a115-4d11-a435-cc22773b668c,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:01.525243 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:01.544818 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bafbec43-79f5-49b7-872f-51e72bc49561 req-1e80aec0-7dbd-45c6-b267-668aed4d47cc tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0958259 May 25 16:07:01.649874 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.651864 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5722980 May 25 16:07:01.655322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45804) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:01.662287 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d69a72d5-419a-489c-b21f-eee55561f332 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:07:01.666557 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:01.678680 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:01.683474 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:01.692100 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:01.713345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:01.831349 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:01.968327 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port d69a72d5-419a-489c-b21f-eee55561f332 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:07:01.979797 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port d69a72d5-419a-489c-b21f-eee55561f332 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:07:01.983493 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.012373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.063501 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dcf15eca-54a1-40d6-9c3c-86f112095199 req-b86ab628-a40f-4675-b92f-820855312096 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:02.072808 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dcf15eca-54a1-40d6-9c3c-86f112095199 req-b86ab628-a40f-4675-b92f-820855312096 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.1701007 May 25 16:07:02.093938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.104064 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-24e2d367-31a6-4c54-abea-f8309892a486 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:02.154211 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=bf44440e-2a5a-4c98-8928-f6f07ea47e68,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:02.163297 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.060s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:02.182397 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dcf15eca-54a1-40d6-9c3c-86f112095199 req-22e92810-412a-4a3d-a8bd-2c055e66dcfb tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1065202 May 25 16:07:02.260965 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.262223 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.6055064 May 25 16:07:02.266580 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45818) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:02.274359 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:02.285757 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:02.294440 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:02.297457 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.304112 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:02.328068 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:02.356817 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port d69a72d5-419a-489c-b21f-eee55561f332, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d69a72d5-419a-489c-b21f-eee55561f332,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:02.410375 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.securitygroups_db [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Project 5d4b9c0f27af44a9ace8862a2133a0cb added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:07:02.419713 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3477c98d-0067-4a7b-961a-5340e9dddc49 req-c9a7ab0f-4dc2-43c9-9cc2-6da4e18c3e49 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0865154 May 25 16:07:02.436716 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.452743 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.456194 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:02.601461 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:02.606411 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.609091 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:02.684296 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:02.687914 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f13ddbe1-89f8-4b6a-9cf3-cfe8858e8d23 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:02.742665 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=ccb38130-a445-4832-a4ae-0354e4a0d8bf,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:02.745855 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:02.857318 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-69469dd4-6d01-440c-abd1-638093360d9a req-4334a573-4c27-48c2-936b-a6e90b71718c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:02.859236 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-69469dd4-6d01-440c-abd1-638093360d9a req-4334a573-4c27-48c2-936b-a6e90b71718c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1293445 May 25 16:07:02.969533 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-69469dd4-6d01-440c-abd1-638093360d9a req-9593b25e-4bfe-4d9a-88d6-85f0a8a7d2ca tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1030943 May 25 16:07:03.244166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.245402 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.9770713 May 25 16:07:03.248674 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45834) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:03.255550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'security_group_rule': {'security_group_id': '503c75ec-7911-4bdd-93c2-bfebd1e51c71', 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:03.263414 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:03.271028 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:security_group_rule and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:03.274992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:03.283990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:03.436128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88640245-17b0-4780-a697-c08fce28a52c req-ef3b9f22-fe8a-40ae-8296-64ba4bead9ff tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:03.437922 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-88640245-17b0-4780-a697-c08fce28a52c req-ef3b9f22-fe8a-40ae-8296-64ba4bead9ff tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.0850770 May 25 16:07:03.490157 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-88640245-17b0-4780-a697-c08fce28a52c req-fc9fe7d4-b392-4965-b8cf-098c50691690 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0453119 May 25 16:07:03.497008 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.securitygroups_db [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Project 5d4b9c0f27af44a9ace8862a2133a0cb added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:07:03.504614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.564160 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d69a72d5-419a-489c-b21f-eee55561f332 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:07:03.602233 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.656339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port d69a72d5-419a-489c-b21f-eee55561f332 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:07:03.656833 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.677704 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fc2b7fa0-39cc-4862-ad82-9010e5ae96cd req-5da1e279-a33d-4113-bb44-382ba9ad0cc7 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0518730 May 25 16:07:03.686594 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.690319 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-37a24ce7-629e-4f14-9345-e9a5818b5409 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:03.718621 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=7ec3a230-d59c-4018-92a5-b3315cd90660,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:03.721734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.032s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:03.811734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:03.815704 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.5633368 May 25 16:07:04.231817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b46ad548-d1a3-4e93-8243-310e885f20a6 req-65f92a7e-68e4-45bc-8e87-b4b5410fc8fb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:04.239635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b46ad548-d1a3-4e93-8243-310e885f20a6 req-65f92a7e-68e4-45bc-8e87-b4b5410fc8fb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1248159 May 25 16:07:04.287512 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b46ad548-d1a3-4e93-8243-310e885f20a6 req-90aeafcb-6a3d-4d9e-8742-5cb7facc89b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0468895 May 25 16:07:04.308489 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Deleting port 3935ec55-2a13-43b8-b0bf-f9ef947051dc {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:07:04.309026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:04.479832 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] For port 3935ec55-2a13-43b8-b0bf-f9ef947051dc, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=3935ec55-2a13-43b8-b0bf-f9ef947051dc,segment=NetworkSegment(be9421e6-1214-4783-b6f4-e62225b27ecf),segment_id=be9421e6-1214-4783-b6f4-e62225b27ecf)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:04.480210 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:04.732423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Calling delete_port for 3935ec55-2a13-43b8-b0bf-f9ef947051dc owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:07:04.755613 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1702668c-d3b6-4625-a714-6e6439216703 req-09928c29-ed20-4a26-8038-59e23ab69c90 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:04.757376 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1702668c-d3b6-4625-a714-6e6439216703 req-09928c29-ed20-4a26-8038-59e23ab69c90 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 944 time: 0.1040609 May 25 16:07:04.813929 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1702668c-d3b6-4625-a714-6e6439216703 req-b8ad9854-1f76-41a3-8e35-6ab5ab82a5ba tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/security-groups?id=651bb4e0-fd9c-4049-a285-3f148efba400&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0457892 May 25 16:07:04.976021 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:04.986662 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-85e2184d-05fa-435e-8038-acf2f14ad1e4 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:05.052250 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Pushing event deleted for resources: {'Port': ['ID=3935ec55-2a13-43b8-b0bf-f9ef947051dc,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:05.058736 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.072s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:05.105159 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a267a1ef-28f8-4853-a510-d4552abfbbb1 req-f48ebb9b-a9af-4158-b922-e58395a603ef tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] 10.222.0.82 "GET /v2.0/ports?device_id=d5ad0bd5-f89c-49d8-bd24-4208d957fb2a HTTP/1.1" status: 200 len: 210 time: 0.0837984 May 25 16:07:05.151692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] No DVR routers for this DVR port 3935ec55-2a13-43b8-b0bf-f9ef947051dc on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:07:05.155869 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:05.159468 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-deleted', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:07:05.241702 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '99fbc72b-cc52-4166-b311-c4e10c3b48de', 'name': 'network-vif-deleted', 'tag': '3935ec55-2a13-43b8-b0bf-f9ef947051dc', 'status': 'completed', 'code': 200} May 25 16:07:05.309400 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-1c6b4e1b-bd2a-4001-8d0d-1064f68c63bf req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.42 "DELETE /v2.0/ports/3935ec55-2a13-43b8-b0bf-f9ef947051dc HTTP/1.1" status: 204 len: 173 time: 1.1582410 May 25 16:07:05.524564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70d2555d-3536-4c0e-a099-7aff73357e4c req-a4b70778-dbdc-497d-b064-3981ac9fc863 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:05.526665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70d2555d-3536-4c0e-a099-7aff73357e4c req-a4b70778-dbdc-497d-b064-3981ac9fc863 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1052225 May 25 16:07:05.598734 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70d2555d-3536-4c0e-a099-7aff73357e4c req-9a29e083-37b8-4083-8537-f867ce6256fc tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0639672 May 25 16:07:05.696889 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-20e77c21-2e6c-4cd9-95a4-50e0f1c7b138 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:05.994810 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6d97e9b8-f195-4620-a0e7-bef026e1c1e6 req-bfd779a7-3cbc-448c-9f95-f49b7391cd3f tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] 10.222.0.82 "GET /v2.0/ports?device_id=99fbc72b-cc52-4166-b311-c4e10c3b48de HTTP/1.1" status: 200 len: 210 time: 0.0466766 May 25 16:07:06.213777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45922) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:06.880169 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2e863966-1f4f-410c-ad0d-0a6e27218699 req-9b2e7197-42e4-424a-b401-d3e799c0aba7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:06.881975 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:06.882835 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2e863966-1f4f-410c-ad0d-0a6e27218699 req-9b2e7197-42e4-424a-b401-d3e799c0aba7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1132586 May 25 16:07:06.957208 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2e863966-1f4f-410c-ad0d-0a6e27218699 req-7525b739-9d97-4b3f-9690-ad42eec0b5ac tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0648727 May 25 16:07:07.102200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45936) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:07.165700 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Deleting subnet b19f7657-6038-4479-b9ec-9e4e92c69bb0 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:07:07.168946 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.230327 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.296892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.434082 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Deleting port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:07:07.434611 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.679025 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] For port 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,segment=NetworkSegment(c781c6c6-6d43-4217-aae6-6935286facf2),segment_id=c781c6c6-6d43-4217-aae6-6935286facf2)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:07.680090 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.799805 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.814385 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:07.848426 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:07.860451 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] 10.222.0.82 "DELETE /v2.0/subnets/b19f7657-6038-4479-b9ec-9e4e92c69bb0 HTTP/1.1" status: 204 len: 168 time: 0.7526138 May 25 16:07:07.861311 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45948) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:07.889131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Pushing event deleted for resources: {'Subnet': ['ID=b19f7657-6038-4479-b9ec-9e4e92c69bb0,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:07.894579 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c9f30476-f985-4490-8c23-01ab458bef9e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:07.948230 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Calling delete_port for 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 owned by network:router_interface {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:07:08.123035 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.160568 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.335717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.353478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['9123c0ac-e64c-41a9-8d68-d98246284959']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:08.369460 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'be9421e6-1214-4783-b6f4-e62225b27ecf') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:08.371173 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'be9421e6-1214-4783-b6f4-e62225b27ecf') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:08.383975 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.405533 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-565f6295-db11-45aa-a4f7-c581d07d2c1c req-93e5f532-4113-4451-8b32-caf5ad13407d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:08.407466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-565f6295-db11-45aa-a4f7-c581d07d2c1c req-93e5f532-4113-4451-8b32-caf5ad13407d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.2749691 May 25 16:07:08.417717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.440020 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.442564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Releasing vlan 948 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:07:08.442994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.446306 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:07:08.457685 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:07:08.496933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Pushing event deleted for resources: {'Port': ['ID=9ca03cbe-10dd-4e7e-a9ef-5a36837d3215,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:08.502804 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c91e5b91-9fad-434d-96ea-d1251e957d99 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.527357 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:08.573702 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-565f6295-db11-45aa-a4f7-c581d07d2c1c req-3b7d0b99-fb77-4a2c-81ca-ebf30e689f04 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1585443 May 25 16:07:08.701256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.709586 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:08.754517 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.768922 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:08.773708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Pushing event deleted for resources: {'Network': ['ID=9123c0ac-e64c-41a9-8d68-d98246284959,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:08.796659 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:08.867874 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-cebafd4f-ed1e-4e68-b73d-6d6dc62c1374 admin admin] 10.222.0.82 "DELETE /v2.0/networks/9123c0ac-e64c-41a9-8d68-d98246284959 HTTP/1.1" status: 204 len: 168 time: 0.9992821 May 25 16:07:09.119991 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:09.226852 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Router d1ce813e-f0ed-493f-8885-b9884ac8f21e has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:07:09.421714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:07:09.432479 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Sync routers for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:07:09.446344 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:07:09.447557 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:07:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:09.458519 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45964) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:09.513423 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] 10.222.0.82 "PUT /v2.0/routers/d1ce813e-f0ed-493f-8885-b9884ac8f21e/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.2977412 May 25 16:07:09.517398 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 45968) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:09.677700 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-57249b0a-86bc-4b9c-84c1-3c5f1cc44e39 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=ed476c44700b47648958b0d2e020320d&name=default HTTP/1.1" status: 200 len: 2975 time: 0.2179098 May 25 16:07:09.681575 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 45976) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:09.694599 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:07:09.756640 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks [] for router, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:09.777875 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:09.809607 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.022430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.028356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1c8f372f-b3d9-46eb-bbdb-bc07eef4bd8c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:10.032101 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:07:08Z', 'revision_number': 5, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:10.036065 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e16a8a69-d9cb-4d56-9e5e-0852ee48b43f req-88ba6c77-bfe2-4d42-b0b6-b46fe918d738 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:10.037195 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Checking router: d1ce813e-f0ed-493f-8885-b9884ac8f21e for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:07:10.043623 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e16a8a69-d9cb-4d56-9e5e-0852ee48b43f req-88ba6c77-bfe2-4d42-b0b6-b46fe918d738 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.2899122 May 25 16:07:10.049061 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=651bb4e0-fd9c-4049-a285-3f148efba400,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:10.053118 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:10.056901 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Routers {'d1ce813e-f0ed-493f-8885-b9884ac8f21e'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:07:10.057858 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 6, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 6, "log_agent_heartbeats": false, "routers": 6},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:07:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:10.063290 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.095129 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e16a8a69-d9cb-4d56-9e5e-0852ee48b43f req-0e45a12a-e7d2-410a-8e7b-eebf610e259d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0508976 May 25 16:07:10.140704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.140704 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/651bb4e0-fd9c-4049-a285-3f148efba400 HTTP/1.1" status: 204 len: 168 time: 0.4568799 May 25 16:07:10.260961 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Deleting port c8688276-a340-4491-aefe-0346cae7a12a {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:07:10.261382 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.288933 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] Set router d1ce813e-f0ed-493f-8885-b9884ac8f21e gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:07:10.393429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] For port c8688276-a340-4491-aefe-0346cae7a12a, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=c8688276-a340-4491-aefe-0346cae7a12a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:10.393893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.502625 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:10.522688 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Calling delete_port for c8688276-a340-4491-aefe-0346cae7a12a owned by network:router_gateway {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:07:10.603581 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:07:08Z', 'revision_number': 5, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:10.615259 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-b8428b2e-1b17-4711-bac3-1c222da77fde None None] The sync data for ids ['d1ce813e-f0ed-493f-8885-b9884ac8f21e'] in dv-cinder-786384-6-iscsi is: [{'id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'name': 'tempest-ServerRescueNegativeTestJSON-275192402-router', 'tenant_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194'}], 'enable_snat': True}, 'gw_port_id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:07:08Z', 'revision_number': 5, 'project_id': 'd20a1b70ac3f4086aac2870ee8c08efe', 'gw_port': {'id': 'c8688276-a340-4491-aefe-0346cae7a12a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:2a:07:f3', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'd1ce813e-f0ed-493f-8885-b9884ac8f21e', 'device_owner': 'network:router_gateway', 'standard_attr_id': 85, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.194', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:03Z', 'updated_at': '2021-05-25T16:03:18Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:07:11.069639 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.093264 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:11.125301 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.259819 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Pushing event deleted for resources: {'Port': ['ID=c8688276-a340-4491-aefe-0346cae7a12a,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:11.262474 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.169s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:11.349183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.578130 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:07:11.588851 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cd9eeeb5-424a-4333-93b8-740a98734cb7 req-d3d5c7d3-4fb2-41cc-8f26-ba2f251cfddc tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:11.590517 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cd9eeeb5-424a-4333-93b8-740a98734cb7 req-d3d5c7d3-4fb2-41cc-8f26-ba2f251cfddc tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.3361881 May 25 16:07:11.592927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.595910 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.626108 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:07:11.626543 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9ca03cbe-10dd-4e7e-a9ef-5a36837d3215 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:07:11.697822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46012) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:11.723873 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cd9eeeb5-424a-4333-93b8-740a98734cb7 req-e4357f94-240d-4af4-a69d-61e3882168b0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1205342 May 25 16:07:11.771139 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.771692 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Fanout notify agent at l3_agent the message router_deleted on router d1ce813e-f0ed-493f-8885-b9884ac8f21e {{(pid=99545) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:07:11.787012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:11.790539 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] 10.222.0.82 "DELETE /v2.0/routers/d1ce813e-f0ed-493f-8885-b9884ac8f21e HTTP/1.1" status: 204 len: 168 time: 2.2694926 May 25 16:07:11.791072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46020) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:11.957050 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-1601860286-network', 'tenant_id': '966c7b0bf03b412dbdfb58e801f9e19a'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:11.965504 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:11.976810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Usage tracker for resource:network and tenant:966c7b0bf03b412dbdfb58e801f9e19a is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:11.981430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:11.990913 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:12.006900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Deleting subnet 051377f2-6b21-4071-b8f5-21107e3c870f {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:07:12.010378 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.073832 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.087981 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.117042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Made reservation on behalf of 966c7b0bf03b412dbdfb58e801f9e19a for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:07:12.119479 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.145163 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.174546 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:12.209873 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Usage tracker for resource:security_group_rule and tenant:966c7b0bf03b412dbdfb58e801f9e19a is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:12.216363 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:12.276537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:12.494916 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.609247 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.612797 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:12.644868 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.648598 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] vlan segment allocate from pool success with {'vlan_id': 901, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:07:12.660194 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=40614cc5-faca-48ce-9412-ffcaac6027bc,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:12.670709 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.058s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:12.671466 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.012s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:12.704658 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.705791 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Added segment 881f52a1-4f7e-475f-8815-9b4741133d22 of type vlan for network 5ed36342-869f-4cf4-9ee8-35103db01053 May 25 16:07:12.717514 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.721536 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=051377f2-6b21-4071-b8f5-21107e3c870f,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:12.730393 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:12.759082 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.762062 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46030) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:12.762616 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/051377f2-6b21-4071-b8f5-21107e3c870f HTTP/1.1" status: 204 len: 168 time: 0.9698055 May 25 16:07:12.869373 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.870911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:12.871733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:12.879150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:12.893555 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.039388 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Pushing event updated for resources: {'Network': ['ID=5ed36342-869f-4cf4-9ee8-35103db01053,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:13.044427 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.165s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:13.055860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 5ed36342-869f-4cf4-9ee8-35103db01053 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:07:13.066930 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-ba936b85-cdcd-440e-a1fb-d481546282b4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:13.066930 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-ba936b85-cdcd-440e-a1fb-d481546282b4 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1097 time: 0.1035461 May 25 16:07:13.095286 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 898 time: 1.3962221 May 25 16:07:13.097523 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46038) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:13.124270 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Request body: {'subnet': {'network_id': '5ed36342-869f-4cf4-9ee8-35103db01053', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-1601860286-subnet', 'tenant_id': '966c7b0bf03b412dbdfb58e801f9e19a', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:13.154576 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.172523 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['feb61d72-e6b4-48c6-bc7c-d61f51987cf9']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:13.194177 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'c781c6c6-6d43-4217-aae6-6935286facf2') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:13.194177 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'c781c6c6-6d43-4217-aae6-6935286facf2') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:13.195043 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:13.211292 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.212066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Usage tracker for resource:subnet and tenant:966c7b0bf03b412dbdfb58e801f9e19a is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:13.217155 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:13.225872 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:13.233223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.246544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Releasing vlan 945 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:07:13.247015 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:07:13.265264 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:07:13.278605 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-c5006967-e144-437f-a202-0d32202d2b47 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:13.280145 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-c5006967-e144-437f-a202-0d32202d2b47 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.1837151 May 25 16:07:13.338221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e56c8384-13d5-4b86-8a55-406313478613 req-75f02e16-00bc-41bb-98d5-63b8117468c4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:13.339853 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e56c8384-13d5-4b86-8a55-406313478613 req-75f02e16-00bc-41bb-98d5-63b8117468c4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.3674259 May 25 16:07:13.345933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Made reservation on behalf of 966c7b0bf03b412dbdfb58e801f9e19a for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:07:13.348731 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Notify callbacks [] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.405356 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e56c8384-13d5-4b86-8a55-406313478613 req-55553533-3b16-4162-9eee-24e977768d82 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0591776 May 25 16:07:13.482318 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-9b22f48e-cd19-45a3-8c63-9d2c0ec8a5cc service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1000 time: 0.1927786 May 25 16:07:13.485509 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.490393 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:13.495446 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:13.523784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Pushing event deleted for resources: {'Network': ['ID=feb61d72-e6b4-48c6-bc7c-d61f51987cf9,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:13.525274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device c8688276-a340-4491-aefe-0346cae7a12a no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:07:13.525405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:13.562945 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: c8688276-a340-4491-aefe-0346cae7a12a {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:07:13.563337 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device c8688276-a340-4491-aefe-0346cae7a12a not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:07:13.566348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-b440512d-2967-45eb-b8ca-5afc7c1b4104 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:13.567992 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-b440512d-2967-45eb-b8ca-5afc7c1b4104 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0754578 May 25 16:07:13.612739 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-3e92a2ad-1f49-4ed7-aa49-042810aa9902 admin admin] 10.222.0.82 "DELETE /v2.0/networks/feb61d72-e6b4-48c6-bc7c-d61f51987cf9 HTTP/1.1" status: 204 len: 168 time: 0.8478904 May 25 16:07:13.730403 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-187fd900-5040-4e83-a493-35d65b289043 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:13.731918 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-187fd900-5040-4e83-a493-35d65b289043 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1332138 May 25 16:07:14.047710 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-5bd9cb31-ae8a-4150-935f-7496cceda4d1 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3063390 May 25 16:07:14.076917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46048) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:14.144923 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5f1f5866-c582-4cd1-bad2-bece4e2f72f3 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=d20a1b70ac3f4086aac2870ee8c08efe&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0644944 May 25 16:07:14.148328 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46052) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:14.218716 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4f1b88af-01a7-4e06-afd9-cf062eaf0b6e req-c9bb509b-6880-48bf-a74e-6422c247cc80 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1617739 May 25 16:07:14.221707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.275942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.307796 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.321486 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:14.411109 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Checking segment: {'id': '881f52a1-4f7e-475f-8815-9b4741133d22', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 901, 'network_id': '5ed36342-869f-4cf4-9ee8-35103db01053'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:07:14.411381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Network hostable DHCP agents. Network: 5ed36342-869f-4cf4-9ee8-35103db01053, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:07:14.430163 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Pushing event updated for resources: {'Subnet': ['ID=51dbf76f-806f-4ed5-863e-1c12233f3340,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:14.442393 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.442857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.443286 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.443713 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.444118 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.444540 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.444949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.445363 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.445835 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.446200 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.446801 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.447522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:14.451952 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.131s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:14.485865 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.491291 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:14.504662 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c255724c-5dbd-46d2-bb27-077184561f8d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:14.509059 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:14.526817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.530595 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/c255724c-5dbd-46d2-bb27-077184561f8d HTTP/1.1" status: 204 len: 168 time: 0.3792541 May 25 16:07:14.536991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Network 5ed36342-869f-4cf4-9ee8-35103db01053 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:07:14.646007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 5ed36342-869f-4cf4-9ee8-35103db01053) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:07:14.651285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:14.655090 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:14.656405 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-df62aa83-b239-471a-adb2-124ac8032c4e None None] Network 5ed36342-869f-4cf4-9ee8-35103db01053 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:07:14.675740 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:07:14.684095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a02c7841-36cc-494d-9da0-2b993a48189e req-06f52ee5-af2e-4f19-982f-5c29507dd8d7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:14.685622 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a02c7841-36cc-494d-9da0-2b993a48189e req-06f52ee5-af2e-4f19-982f-5c29507dd8d7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 944 time: 0.1243393 May 25 16:07:14.854595 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a02c7841-36cc-494d-9da0-2b993a48189e req-94bfb936-445e-4129-a77d-1efd17b15a35 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1577978 May 25 16:07:14.869649 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-cbdd4755-323f-467e-b708-f833b7867bec admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 862 time: 1.7709064 May 25 16:07:15.055864 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-71c376ca-df2b-4cab-bb34-1a670a928d66 None None] Network 5ed36342-869f-4cf4-9ee8-35103db01053 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:07:15.154406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:15.185452 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'mac_address': 'fa:16:3e:fe:cd:1b', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.12'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:07:15.263759 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:07:15.278418 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:15.313104 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:07:15.395499 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:15.533110 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:15.540508 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:15.639351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=9']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:15.645536 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:15.895286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['5df9097f-ef39-4cbc-b791-fce24729f2ed'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:07:15.933222 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46114) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:15.957806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Request body: {'network': {'name': 'tempest-EncryptedVolumesExtendAttachedTest-855163410-network', 'tenant_id': '581e0208e8204b699743a125dae2aab9'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:15.968903 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:15.973991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Usage tracker for resource:network and tenant:581e0208e8204b699743a125dae2aab9 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:15.978816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:15.985083 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:16.072704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Made reservation on behalf of 581e0208e8204b699743a125dae2aab9 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:07:16.075305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.094142 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.116737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:16.123654 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Usage tracker for resource:security_group_rule and tenant:581e0208e8204b699743a125dae2aab9 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:16.127035 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:16.135423 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:16.165949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6df1df1-4c01-4480-8a8e-6a5b6fd96347 req-b8a4ff54-4c3e-42f8-bcd4-4f121b633d00 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:16.167621 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6df1df1-4c01-4480-8a8e-6a5b6fd96347 req-b8a4ff54-4c3e-42f8-bcd4-4f121b633d00 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1465280 May 25 16:07:16.233234 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6df1df1-4c01-4480-8a8e-6a5b6fd96347 req-d1beb94b-865f-407b-924b-84f452580a80 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0591772 May 25 16:07:16.290246 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.450729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.454994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d4af9b5c-a4a1-42ab-abf8-3fa5e6b98095 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:16.478575 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] vlan segment allocate from pool success with {'vlan_id': 921, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:07:16.491465 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5ef1839a-1638-4433-90f1-13c46005f67a,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:16.495366 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:16.496287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.496613 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Added segment 8816aea5-313b-43ab-99fd-958892f53776 of type vlan for network b4fa853a-3837-4de3-a49b-00ea9e04ee2d May 25 16:07:16.529247 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.637335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.639227 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:16.639394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:16.644608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:16.753173 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Pushing event updated for resources: {'Network': ['ID=b4fa853a-3837-4de3-a49b-00ea9e04ee2d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:16.756252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.112s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:16.764543 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network b4fa853a-3837-4de3-a49b-00ea9e04ee2d details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:07:16.857479 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 897 time: 0.9225242 May 25 16:07:16.859916 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46144) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:16.866129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Request body: {'subnet': {'network_id': 'b4fa853a-3837-4de3-a49b-00ea9e04ee2d', 'cidr': '172.20.0.0/28', 'name': 'tempest-EncryptedVolumesExtendAttachedTest-855163410-subnet', 'tenant_id': '581e0208e8204b699743a125dae2aab9', 'enable_dhcp': False, 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:16.872628 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:16.875698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Usage tracker for resource:subnet and tenant:581e0208e8204b699743a125dae2aab9 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:16.878980 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:16.884667 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:16.959667 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Made reservation on behalf of 581e0208e8204b699743a125dae2aab9 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:07:16.962325 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:17.471145 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77bf7794-404e-4fc0-8844-9fb173413058 req-73ba21b3-ea56-4b08-9c76-3aaef508372c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:17.472661 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77bf7794-404e-4fc0-8844-9fb173413058 req-73ba21b3-ea56-4b08-9c76-3aaef508372c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1087527 May 25 16:07:17.527205 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77bf7794-404e-4fc0-8844-9fb173413058 req-5fed1786-c18c-4bf5-b365-b6fac9587455 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0486701 May 25 16:07:17.583817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:17.587611 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9690a580-eb02-4c49-bcec-a1aa10417053 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:17.657600 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Pushing event updated for resources: {'Subnet': ['ID=5cb38515-3972-4181-886e-d99af60c4bf9,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:17.661399 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:17.668854 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Checking segment: {'id': '8816aea5-313b-43ab-99fd-958892f53776', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 921, 'network_id': 'b4fa853a-3837-4de3-a49b-00ea9e04ee2d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:07:17.669210 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Network hostable DHCP agents. Network: b4fa853a-3837-4de3-a49b-00ea9e04ee2d, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:07:17.747905 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Network b4fa853a-3837-4de3-a49b-00ea9e04ee2d is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:07:17.873572 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, b4fa853a-3837-4de3-a49b-00ea9e04ee2d) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:07:17.883060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:17.883231 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-93bbfb9f-2922-4973-881f-99c7550ba170 None None] Network b4fa853a-3837-4de3-a49b-00ea9e04ee2d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:07:17.885030 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:17.972603 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-18c8e1ed-f521-4368-a6d6-c8d41ab0ae08 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 861 time: 1.1115625 May 25 16:07:18.084240 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-9919a7ad-be2c-4d7a-9e50-4b9ca6093ebd None None] Network b4fa853a-3837-4de3-a49b-00ea9e04ee2d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:07:18.116960 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46164) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:18.179307 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Deleting subnet 51dbf76f-806f-4ed5-863e-1c12233f3340 {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:07:18.182768 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.238422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Notify callbacks [] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.245744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.582490 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.591941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3b882c73-4216-42e0-a264-732d58e8c39a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:18.635425 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Pushing event deleted for resources: {'Subnet': ['ID=51dbf76f-806f-4ed5-863e-1c12233f3340,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:18.639654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:18.641500 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.648131 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-64d35f4d-de61-414b-91ec-7921e073bbed admin admin] 10.222.0.82 "DELETE /v2.0/subnets/51dbf76f-806f-4ed5-863e-1c12233f3340 HTTP/1.1" status: 204 len: 168 time: 0.5299931 May 25 16:07:18.650493 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46176) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:18.738565 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ac7db19d-81f3-49d6-8252-2fe4b269ac52 req-6d2ee173-9e8f-48a9-a21e-4c425892c9ef tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:18.740066 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ac7db19d-81f3-49d6-8252-2fe4b269ac52 req-6d2ee173-9e8f-48a9-a21e-4c425892c9ef tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.0899405 May 25 16:07:18.758625 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.787389 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ac7db19d-81f3-49d6-8252-2fe4b269ac52 req-811b52ca-f026-4ac9-a0a4-c06f41b1b6c5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0413377 May 25 16:07:18.893125 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:18.909245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > segment before_delete 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.909534 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet precommit_delete_associations 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.910256 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > segment_host_mapping after_create 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.910587 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > port after_create 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.910945 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > port after_delete 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.911328 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > port after_update 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.911648 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > segment after_delete 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.911961 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.912272 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.912586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet after_update 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.993748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet after_create 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.994043 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet after_delete 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.994377 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet before_create 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.994707 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Subscribe: > subnet before_update 55550000 {{(pid=99544) subscribe /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:45}} May 25 16:07:18.995098 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['5ed36342-869f-4cf4-9ee8-35103db01053']}} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:19.009049 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '881f52a1-4f7e-475f-8815-9b4741133d22') {'for_net_delete': True} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:19.009488 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '881f52a1-4f7e-475f-8815-9b4741133d22') {} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:19.021579 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3244641'] for segment, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.037548 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.044389 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Releasing vlan 901 on physical network physnet1 to pool {{(pid=99544) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:07:19.044775 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-3070910'] for segment, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: from cryptography.utils import int_from_bytes May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: from cryptography.utils import int_from_bytes May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: Traceback (most recent call last): May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: timer() May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: cb(*args, **kw) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: return func(*args, **kwargs) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: self._notify() May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: self.callback(batched_events) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: event.method(event) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: return f(self, *a, **k) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._get(url).json() May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: return self.request(url, 'GET', **kwargs) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:07:19.060273 dv-cinder-786384-6-iscsi neutron-server[99544]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:07:19.243420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.253605 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.259048 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:19.276147 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] 10.222.0.82 "DELETE /v2.0/networks/5ed36342-869f-4cf4-9ee8-35103db01053 HTTP/1.1" status: 204 len: 168 time: 0.6246219 May 25 16:07:19.279110 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46184) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:19.281965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Pushing event deleted for resources: {'Network': ['ID=5ed36342-869f-4cf4-9ee8-35103db01053,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:19.281965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:19.341236 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Deleting subnet 5cb38515-3972-4181-886e-d99af60c4bf9 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:07:19.344707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.399418 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.456363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.517719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46190) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:19.680058 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.697703 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:19.703611 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Usage tracker for resource:security_group_rule and tenant:aa744fd9abc24640a4b50bdf2e896516 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:19.709452 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:19.714669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:19.828560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.838594 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:19.904150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=5cb38515-3972-4181-886e-d99af60c4bf9,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:19.910251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.911384 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.919074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.919436 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.919992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.920502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.921032 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.921527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.922029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.922535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.923128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.923619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.924106 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.925455 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:07:19.929851 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.091s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:19.932009 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/5cb38515-3972-4181-886e-d99af60c4bf9 HTTP/1.1" status: 204 len: 168 time: 0.6518314 May 25 16:07:19.934430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46208) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:19.985933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:19.995550 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:20.044318 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=aa744fd9abc24640a4b50bdf2e896516&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5244055 May 25 16:07:20.046456 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=7d74bee4-c766-4fe3-a88f-4b2a5f6c4920,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:20.050760 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46212) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:20.054965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:20.119442 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.153624 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.215725 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.294679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.307236 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c1724d73-17e6-4d65-96fb-6e892c1a6d2c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:20.323453 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=7d74bee4-c766-4fe3-a88f-4b2a5f6c4920,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:20.325109 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:20.326392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.327810 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/7d74bee4-c766-4fe3-a88f-4b2a5f6c4920 HTTP/1.1" status: 204 len: 168 time: 0.2757714 May 25 16:07:20.379111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ad212a5f-44de-4884-94b1-50e5a75b1b0a req-26bd9029-38f0-4520-af2a-0c4d698f792d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:20.388355 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ad212a5f-44de-4884-94b1-50e5a75b1b0a req-26bd9029-38f0-4520-af2a-0c4d698f792d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.4325361 May 25 16:07:20.429013 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.451441 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['b4fa853a-3837-4de3-a49b-00ea9e04ee2d']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:20.469970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '8816aea5-313b-43ab-99fd-958892f53776') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:20.470276 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '8816aea5-313b-43ab-99fd-958892f53776') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:07:20.484022 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ad212a5f-44de-4884-94b1-50e5a75b1b0a req-4d5a37a1-60b5-49cc-bfc8-2042325c9081 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0789287 May 25 16:07:20.487774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.506056 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.512918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Releasing vlan 921 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:07:20.513322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:07:20.528097 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:07:20.756837 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.762082 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-edadbcc4-b118-4f4b-9502-b8403768b516 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:20.777479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:20.787988 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Pushing event deleted for resources: {'Network': ['ID=b4fa853a-3837-4de3-a49b-00ea9e04ee2d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:20.790664 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] 10.222.0.82 "DELETE /v2.0/networks/b4fa853a-3837-4de3-a49b-00ea9e04ee2d HTTP/1.1" status: 204 len: 168 time: 0.8548508 May 25 16:07:20.791887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3a91c8c6-d30c-473b-85a5-9192369134c6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:21.217153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46232) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:21.235697 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.251071 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:21.255383 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Usage tracker for resource:security_group_rule and tenant:7085d1ae21d54b78822d386ad7690182 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:21.258271 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:21.264816 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:21.297936 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46236) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:21.351628 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-dcc89247-3793-43f4-b30e-87077e6e9ee2 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=966c7b0bf03b412dbdfb58e801f9e19a&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0503051 May 25 16:07:21.351941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46242) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:21.354477 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.417942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.439181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.445572 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.448731 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-81b1ecb0-54f6-421d-9e9c-76f6c1fece61 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:21.489453 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=7085d1ae21d54b78822d386ad7690182&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2711856 May 25 16:07:21.490392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f5d03155-732d-4e74-a04b-3ab1b4921024,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:21.492843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46246) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:21.493402 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:21.554198 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.581473 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.593350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.603146 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1f308830-3f18-4d56-a177-16e26a36bb51 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:21.659404 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=40614cc5-faca-48ce-9412-ffcaac6027bc,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:21.672149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.069s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:21.691896 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.692973 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/40614cc5-faca-48ce-9412-ffcaac6027bc HTTP/1.1" status: 204 len: 168 time: 0.3401461 May 25 16:07:21.730251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fba06675-ca69-4864-a924-54792449d3ae req-e5e2d500-aa5c-45f9-8792-d1badda2e9ff tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:21.731847 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fba06675-ca69-4864-a924-54792449d3ae req-e5e2d500-aa5c-45f9-8792-d1badda2e9ff tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1177776 May 25 16:07:21.787828 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fba06675-ca69-4864-a924-54792449d3ae req-f9d40359-4aeb-4d2e-bab8-6dc6c4f47824 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0493147 May 25 16:07:21.825908 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.832340 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:21.847537 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f5d03155-732d-4e74-a04b-3ab1b4921024,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:21.850690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:21.852666 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:21.853854 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f5d03155-732d-4e74-a04b-3ab1b4921024 HTTP/1.1" status: 204 len: 168 time: 0.3603067 May 25 16:07:22.564989 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46274) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:22.620842 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e4e57b6b-890d-47b3-8ca1-862f1e8802e5 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=581e0208e8204b699743a125dae2aab9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0545480 May 25 16:07:22.623407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46278) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:22.691634 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:22.718849 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:22.854842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:22.860716 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a026d586-d51a-45ea-87eb-61b9defd0b0b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:22.875135 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5ef1839a-1638-4433-90f1-13c46005f67a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:22.877890 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:22.886886 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:22.890623 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5ef1839a-1638-4433-90f1-13c46005f67a HTTP/1.1" status: 204 len: 168 time: 0.2636650 May 25 16:07:22.901438 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:07:22.993854 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 5df9097f-ef39-4cbc-b791-fce24729f2ed triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:07:22.994230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:23.018663 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2d247b3b-6db3-4b2c-b4f5-d0ae972c9a8b req-6ae9cfa4-0949-43b2-9159-7f61f58987ac tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:23.020342 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2d247b3b-6db3-4b2c-b4f5-d0ae972c9a8b req-6ae9cfa4-0949-43b2-9159-7f61f58987ac tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 945 time: 0.1149127 May 25 16:07:23.070996 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2d247b3b-6db3-4b2c-b4f5-d0ae972c9a8b req-db619d4b-b617-439c-ad63-ab824aaf1713 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0434034 May 25 16:07:23.208166 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:23.263801 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:07:23.277210 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:23.329142 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'code': 200} May 25 16:07:23.377363 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:23.381358 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:23.468214 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=10']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:23.473135 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:24.328069 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4be6c9d9-edd7-4e85-bd12-120110b6e9df req-68781d70-e42d-4332-814a-c740e174e7e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:24.329933 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4be6c9d9-edd7-4e85-bd12-120110b6e9df req-68781d70-e42d-4332-814a-c740e174e7e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1237659 May 25 16:07:24.386568 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4be6c9d9-edd7-4e85-bd12-120110b6e9df req-6cab279f-7dad-4e00-8296-544aa9c6298e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0500319 May 25 16:07:24.872330 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-59052fa9-ce56-4f2d-a8a9-199a43aabf8d req-a6c6578f-0522-4b9d-ab55-48559a0dcad1 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:24.874290 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-59052fa9-ce56-4f2d-a8a9-199a43aabf8d req-a6c6578f-0522-4b9d-ab55-48559a0dcad1 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1100 time: 0.1894646 May 25 16:07:25.617883 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96bafbb0-a267-417d-9ac0-2803412ea9b5 req-28a76e4f-1593-4fa1-b311-99ec1bb81d02 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:25.620295 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96bafbb0-a267-417d-9ac0-2803412ea9b5 req-28a76e4f-1593-4fa1-b311-99ec1bb81d02 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1020722 May 25 16:07:25.671289 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96bafbb0-a267-417d-9ac0-2803412ea9b5 req-a994f7fd-68df-4984-a3c0-9d98e8f6b26d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0455141 May 25 16:07:25.731759 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-59052fa9-ce56-4f2d-a8a9-199a43aabf8d req-a34abe50-2e09-4d37-8e47-34760b3f2d8e service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0470080 May 25 16:07:26.616692 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46430) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:26.740511 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:26.756923 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:26.761559 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Usage tracker for resource:security_group_rule and tenant:87358ba7c9e24f1bad78a7448f3b5c12 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:26.764034 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:26.770060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:26.924025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9413b01-fc3d-40f6-a830-b08c332b0046 req-1a2153f5-a28f-4fed-9644-d0c373e0d1c8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:26.926058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e9413b01-fc3d-40f6-a830-b08c332b0046 req-1a2153f5-a28f-4fed-9644-d0c373e0d1c8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1343803 May 25 16:07:26.930365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:26.975071 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e9413b01-fc3d-40f6-a830-b08c332b0046 req-42d78788-42cd-493d-825d-f32b66d0770a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431039 May 25 16:07:27.007978 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:27.014706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-15236557-c83e-471e-8798-be0641f767e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:27.056503 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=87358ba7c9e24f1bad78a7448f3b5c12&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4387083 May 25 16:07:27.057691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bf88ee7b-aa18-4b59-ae4b-5d20b59a0f6c,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:27.059124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46438) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:27.060806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:27.125174 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:27.151015 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:27.279511 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:27.283420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b96d6658-03ad-44db-8597-73cbd812c167 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:27.297281 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bf88ee7b-aa18-4b59-ae4b-5d20b59a0f6c,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:27.300850 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:27.302953 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:27.305718 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/bf88ee7b-aa18-4b59-ae4b-5d20b59a0f6c HTTP/1.1" status: 204 len: 168 time: 0.2439024 May 25 16:07:28.066133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46456) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:28.085492 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.101161 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:28.104943 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Usage tracker for resource:security_group_rule and tenant:0700f3e26013482eae86f448ea586e1b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:28.107896 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:28.115091 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:28.171431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4faedb5a-da26-4d38-aa1c-7621097b0bda req-e0f43795-d9e2-4556-a78e-4e3fe53678ae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:28.173029 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4faedb5a-da26-4d38-aa1c-7621097b0bda req-e0f43795-d9e2-4556-a78e-4e3fe53678ae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0831609 May 25 16:07:28.213834 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.221905 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4faedb5a-da26-4d38-aa1c-7621097b0bda req-8465c3cd-f24b-4674-9ce9-9346e775512c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0411921 May 25 16:07:28.288304 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.295858 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1b74f774-8fc5-4fb0-8df8-407024db1e9d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:28.339775 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8aedd5cd-66dc-4581-9d84-41690b18b860,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:28.343282 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:28.346396 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ea85502c-47ac-47ca-804f-7d152b9ab74a admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=0700f3e26013482eae86f448ea586e1b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2789531 May 25 16:07:28.349062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46464) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:28.411251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.438457 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.599191 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.603145 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9cd0f3a2-fe84-451e-a3cd-282940ea07ce admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:28.616106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8aedd5cd-66dc-4581-9d84-41690b18b860,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:28.619142 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:28.629166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:28.629166 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/8aedd5cd-66dc-4581-9d84-41690b18b860 HTTP/1.1" status: 204 len: 168 time: 0.2734902 May 25 16:07:29.422181 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bef43e39-2d22-41e2-9e1b-f47187f1d3dc req-8701f57a-d509-4232-a507-1c0ecbf6c59e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:29.423677 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bef43e39-2d22-41e2-9e1b-f47187f1d3dc req-8701f57a-d509-4232-a507-1c0ecbf6c59e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0852566 May 25 16:07:29.473589 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bef43e39-2d22-41e2-9e1b-f47187f1d3dc req-f57cbacd-1054-43c3-95bb-31b038143dda tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0432112 May 25 16:07:30.689264 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-14983dd1-b39c-4592-bec9-963dbccfcb34 req-eb345ebc-97c2-4476-a77e-ee6a6495ff48 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:30.691075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-14983dd1-b39c-4592-bec9-963dbccfcb34 req-eb345ebc-97c2-4476-a77e-ee6a6495ff48 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0840795 May 25 16:07:30.759964 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-14983dd1-b39c-4592-bec9-963dbccfcb34 req-57d0673a-3aa7-4f44-a826-ebf69003db53 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0614567 May 25 16:07:32.036665 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3f6fb7b4-f906-4aff-9fbf-b276f2b15064 req-eb975c81-5b18-47f6-aa7b-d09ce4e7a050 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:32.038336 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3f6fb7b4-f906-4aff-9fbf-b276f2b15064 req-eb975c81-5b18-47f6-aa7b-d09ce4e7a050 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1260726 May 25 16:07:32.116639 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3f6fb7b4-f906-4aff-9fbf-b276f2b15064 req-24135445-a0cd-4cec-a0d6-dc1dc7a26259 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0683935 May 25 16:07:33.382020 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5d5ee185-4e1b-49ff-9eca-72ee10edffdb req-4914e028-df62-418a-8851-834969fdb537 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:33.384386 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5d5ee185-4e1b-49ff-9eca-72ee10edffdb req-4914e028-df62-418a-8851-834969fdb537 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1060472 May 25 16:07:33.552008 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5d5ee185-4e1b-49ff-9eca-72ee10edffdb req-da65dd88-eedc-4f55-9ee8-8d384081b88d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1614032 May 25 16:07:34.746071 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-52f899b5-1d07-40b8-b656-fef4dc211988 req-00e698c1-2c37-44a2-bb79-48bf4fb65ab8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:34.747543 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-52f899b5-1d07-40b8-b656-fef4dc211988 req-00e698c1-2c37-44a2-bb79-48bf4fb65ab8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0818629 May 25 16:07:34.795076 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-52f899b5-1d07-40b8-b656-fef4dc211988 req-cc012b7e-23a2-421b-b3bd-e7ad6fc51265 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0417519 May 25 16:07:35.646773 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4e17d523-3f4f-4439-9338-79b72b339302 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:35.997366 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-380ce4f5-7b51-462b-8b5a-3a8bc8bf3dab req-b722247f-8a65-4e4b-ab95-0fd1663580ef tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:35.998797 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-380ce4f5-7b51-462b-8b5a-3a8bc8bf3dab req-b722247f-8a65-4e4b-ab95-0fd1663580ef tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0943522 May 25 16:07:36.042189 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-380ce4f5-7b51-462b-8b5a-3a8bc8bf3dab req-f0941b38-69cd-42cb-af06-3bd8ef175400 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0375507 May 25 16:07:37.243971 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-04576d66-1c24-4d06-a2d0-929372a173b7 req-28eea64a-eb1f-4a56-8eb0-3e96c1ac3c5a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:37.245466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-04576d66-1c24-4d06-a2d0-929372a173b7 req-28eea64a-eb1f-4a56-8eb0-3e96c1ac3c5a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0875974 May 25 16:07:37.292492 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-04576d66-1c24-4d06-a2d0-929372a173b7 req-7d960eef-ded0-4f85-a6ee-2407dad8a323 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0409157 May 25 16:07:38.139143 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:38.434290 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-87f2db7d-3d17-4ab0-bdd0-38337972f8c0 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:38.505348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32672cce-6e1f-40ee-982d-de2a70e8042a req-dcd9ee84-3665-4def-a0d6-e1ab2dcde29c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:38.507010 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32672cce-6e1f-40ee-982d-de2a70e8042a req-dcd9ee84-3665-4def-a0d6-e1ab2dcde29c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0966606 May 25 16:07:38.553444 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32672cce-6e1f-40ee-982d-de2a70e8042a req-06de253d-9849-4f23-b650-38afc0c6e090 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0406504 May 25 16:07:39.067787 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:39.797599 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25659ca4-f02f-4701-b8dd-adcaffe0f962 req-6c784218-a372-4ff7-8e53-09a441538337 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:39.801628 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25659ca4-f02f-4701-b8dd-adcaffe0f962 req-6c784218-a372-4ff7-8e53-09a441538337 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1113698 May 25 16:07:39.859523 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25659ca4-f02f-4701-b8dd-adcaffe0f962 req-771ee165-5c5d-459a-837f-794b172830c0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0464306 May 25 16:07:40.419877 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-c7235eed-4c7c-49d4-a4b4-f863c68db1e9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Synchronizing usage tracker for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:07:40.511920 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:40.553417 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-c7235eed-4c7c-49d4-a4b4-f863c68db1e9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Unset dirty status for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:07:40.557046 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-c7235eed-4c7c-49d4-a4b4-f863c68db1e9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1065377766&fields=id&tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb HTTP/1.1" status: 200 len: 267 time: 0.1834724 May 25 16:07:40.724014 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-c60e37c1-e94a-40d3-a80a-5bbfcd3c6606 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:40.724694 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-c60e37c1-e94a-40d3-a80a-5bbfcd3c6606 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Synchronizing usage tracker for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:07:40.850529 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-c60e37c1-e94a-40d3-a80a-5bbfcd3c6606 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Unset dirty status for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:07:40.851736 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-c60e37c1-e94a-40d3-a80a-5bbfcd3c6606 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 809 time: 0.2886102 May 25 16:07:40.867421 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-7045f84c-d46a-4e19-aa1f-a3df13ea31fb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/quotas/5d4b9c0f27af44a9ace8862a2133a0cb HTTP/1.1" status: 200 len: 375 time: 0.0096092 May 25 16:07:40.970594 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-a1d794ea-73e3-46e0-a1f2-5a838becd01f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&fields=id HTTP/1.1" status: 200 len: 302 time: 0.0964997 May 25 16:07:41.133512 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:41.179594 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-8882b1c7-b99b-4558-9e64-b733147e94f9 service neutron] 10.222.0.82 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1659112 May 25 16:07:41.273873 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-211254fd-c150-4c9e-9908-e7d53c1d63c8 req-fbd12b87-0163-4a27-b2ed-b0557f3fe51b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:41.275629 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-211254fd-c150-4c9e-9908-e7d53c1d63c8 req-fbd12b87-0163-4a27-b2ed-b0557f3fe51b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2954500 May 25 16:07:41.350315 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-211254fd-c150-4c9e-9908-e7d53c1d63c8 req-cdc053dd-e821-4302-b85d-5c5e67323fae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0667415 May 25 16:07:41.372846 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-e9e19fc3-b032-4083-ae03-212e1e37e5c7 service neutron] 10.222.0.82 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1842508 May 25 16:07:42.230487 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-272d961d-040f-4e1c-b7d7-5811cf9f5069 req-e149e96d-4e70-46e8-89bd-0aa8d94315d3 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.0530286 May 25 16:07:42.546377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-be5b5220-5da3-49b7-9b04-6561cc8b6591 req-0ceaa4e1-70fa-4127-bf37-bf67d0c400b0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:42.547837 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-be5b5220-5da3-49b7-9b04-6561cc8b6591 req-0ceaa4e1-70fa-4127-bf37-bf67d0c400b0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0822895 May 25 16:07:42.594729 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-be5b5220-5da3-49b7-9b04-6561cc8b6591 req-7b1f30bd-d9aa-4f18-835a-aeac9ea12ad3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0408158 May 25 16:07:43.395003 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b555f729-b598-4bda-a2f0-75d93c15ce98 req-a6dc9bba-26bc-4e23-8659-6d1c6a6bee68 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.0955188 May 25 16:07:43.500348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3140448c-2ae4-482e-a05d-ba005c0a819f req-d2c1e9d6-3d0a-48c3-a50a-1e8d66279217 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:43.501857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3140448c-2ae4-482e-a05d-ba005c0a819f req-d2c1e9d6-3d0a-48c3-a50a-1e8d66279217 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 0.0865886 May 25 16:07:43.550267 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3140448c-2ae4-482e-a05d-ba005c0a819f req-0920d8f9-00b3-4e09-a6f4-9945c4082ddd tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0411656 May 25 16:07:43.798663 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-920f7dcf-c508-4500-be42-dd6c2fd9d8f0 req-a97c06a8-b5ee-4de7-be77-7942f522a02b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:43.801305 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-920f7dcf-c508-4500-be42-dd6c2fd9d8f0 req-a97c06a8-b5ee-4de7-be77-7942f522a02b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0970182 May 25 16:07:43.851140 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-920f7dcf-c508-4500-be42-dd6c2fd9d8f0 req-d0458a1e-9bb7-40b2-877f-d6ea7e0c2534 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0429981 May 25 16:07:44.555682 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-62c2f66a-3c10-4b91-b7bb-ad55569c97a4 req-f72f7a50-7c07-4731-aabf-c7dbdcc11d1b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.0473442 May 25 16:07:44.766528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-13f446ad-2c93-44a1-b31a-f49bc755d551 req-d1b02555-83ca-4cfb-bdaf-1187463faac4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:44.768112 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-13f446ad-2c93-44a1-b31a-f49bc755d551 req-d1b02555-83ca-4cfb-bdaf-1187463faac4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 0.0952652 May 25 16:07:44.993565 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-13f446ad-2c93-44a1-b31a-f49bc755d551 req-ac2ded62-04a5-4147-ba80-f8c57a3b9e8c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.2136071 May 25 16:07:45.093447 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46880) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:45.135233 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea213d6a-3b37-454f-bda7-f0193c0c683b req-4851d6d7-5bb5-4e3b-8abd-7e01c22fd5fc tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:45.143258 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ea213d6a-3b37-454f-bda7-f0193c0c683b req-4851d6d7-5bb5-4e3b-8abd-7e01c22fd5fc tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1033716 May 25 16:07:45.195472 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ea213d6a-3b37-454f-bda7-f0193c0c683b req-3ce8cdf6-79e9-4c9e-8a2b-203ab5b040ae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0510485 May 25 16:07:45.288289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.308726 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:45.314109 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Usage tracker for resource:security_group_rule and tenant:a38bc75bceec456bb4437fe3508f2794 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:45.317984 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:45.325491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:45.361596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38997344-afce-49e6-9604-6fda656fd324 req-3f017c0f-db0e-40d4-ad70-b6c3ecca0e01 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:45.363109 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-38997344-afce-49e6-9604-6fda656fd324 req-3f017c0f-db0e-40d4-ad70-b6c3ecca0e01 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 0.1635318 May 25 16:07:45.574550 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.589043 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Deleting port f1cbd3dd-0666-4b74-80e3-36426f4ec54f {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:07:45.589503 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.602464 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-6435a4b5-9eea-4c76-8fe0-72d157f3c6af tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:45.604134 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-6435a4b5-9eea-4c76-8fe0-72d157f3c6af tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 809 time: 0.1823139 May 25 16:07:45.672078 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-ab94935f-d26c-44c7-a652-990051ef8526 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb HTTP/1.1" status: 200 len: 380 time: 0.0501940 May 25 16:07:45.684806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'port': {'device_id': 'ee7483db-c622-4701-aa41-45aa235c8251', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'admin_state_up': True, 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'security_groups': ['503c75ec-7911-4bdd-93c2-bfebd1e51c71']}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:45.717466 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.728765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-de0e5146-56d6-499a-b306-c858c7660d51 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:45.811746 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=a38bc75bceec456bb4437fe3508f2794&name=default HTTP/1.1" status: 200 len: 2979 time: 0.7088084 May 25 16:07:45.812053 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46894) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:45.816027 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] For port f1cbd3dd-0666-4b74-80e3-36426f4ec54f, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:07:45.816027 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.823809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=34c55db3-ea2c-443d-b76e-5d775e62a2b0,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:45.831596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:45.928081 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:45.935018 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4814fbe0-e6d5-4035-80fc-475561e1f186 req-2419dd67-606a-4dd8-b2bf-44febf32eebf tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.0666611 May 25 16:07:45.937055 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:port and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:45.943036 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:45.953035 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:45.978716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:45.981285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Calling delete_port for f1cbd3dd-0666-4b74-80e3-36426f4ec54f owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:07:46.010015 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.051739 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Made reservation on behalf of 5d4b9c0f27af44a9ace8862a2133a0cb for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:07:46.052137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.219181 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e24d2590-431a-430c-87a3-2862a2f2fd06 req-5ab67b9c-9a02-4015-8564-04c57b80e708 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:46.220928 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e24d2590-431a-430c-87a3-2862a2f2fd06 req-5ab67b9c-9a02-4015-8564-04c57b80e708 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 946 time: 0.0990589 May 25 16:07:46.262822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.269412 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.272052 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0124cb78-9f5e-4850-9326-ad87f027bf38 tempest-VolumesActionsTest-595927417 tempest-VolumesActionsTest-595927417-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:46.284776 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04eae8f6-93b4-41e9-9f32-1105aa93c045 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:46.303211 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e24d2590-431a-430c-87a3-2862a2f2fd06 req-cb4375f8-2f1b-465a-9771-b5191b0ab39f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0759480 May 25 16:07:46.306100 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['f1cbd3dd-0666-4b74-80e3-36426f4ec54f'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:07:46.310506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=34c55db3-ea2c-443d-b76e-5d775e62a2b0,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:46.327147 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.042s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:46.360701 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Pushing event deleted for resources: {'Port': ['ID=f1cbd3dd-0666-4b74-80e3-36426f4ec54f,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:46.364406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.092s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:46.407211 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.408461 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/34c55db3-ea2c-443d-b76e-5d775e62a2b0 HTTP/1.1" status: 204 len: 168 time: 0.5953481 May 25 16:07:46.456485 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] No DVR routers for this DVR port f1cbd3dd-0666-4b74-80e3-36426f4ec54f on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:07:46.461032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.464892 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'name': 'network-vif-deleted', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:07:46.510213 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Allocated IP 172.20.0.7 (6c217a71-5586-47ee-ab1e-21c28b2d40a1/2b5e1fe5-3648-40fc-a046-12baa71ed42b/db215e56-e99f-4ad2-a152-51e2aa8bb9ad) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:07:46.519512 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'cc5cddcb-4e4b-43a8-90e7-165b9b1045ac', 'name': 'network-vif-deleted', 'tag': 'f1cbd3dd-0666-4b74-80e3-36426f4ec54f', 'status': 'completed', 'code': 200} May 25 16:07:46.524491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c99f24a8-54a1-4b5a-b6bd-59d4c4b2d962 req-08eaec63-ccdf-4ce6-a076-9b0f85ff2c5a tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:46.532657 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c99f24a8-54a1-4b5a-b6bd-59d4c4b2d962 req-08eaec63-ccdf-4ce6-a076-9b0f85ff2c5a tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.2168896 May 25 16:07:46.671858 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c99f24a8-54a1-4b5a-b6bd-59d4c4b2d962 req-8e883bce-b79d-41a3-bab5-192625024106 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1382961 May 25 16:07:46.709218 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-38997344-afce-49e6-9604-6fda656fd324 req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "DELETE /v2.0/ports/f1cbd3dd-0666-4b74-80e3-36426f4ec54f HTTP/1.1" status: 204 len: 173 time: 1.3389566 May 25 16:07:46.784667 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.802148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Transition to ACTIVE for port object db215e56-e99f-4ad2-a152-51e2aa8bb9ad will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:07:46.926780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:46.937001 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d4f0a53-7cee-4c50-a7e7-ab9c58716484 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:47.078547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:47.087299 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.150s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:47.102802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:07:47.104461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:07:47.127834 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:07:47.269958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:47.270634 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:47.312521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d5a4a83-4af2-4443-97bf-c8f023f66aa0 req-b79750cd-f415-4ed5-a024-042b7cb8a92e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:47.313248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0d5a4a83-4af2-4443-97bf-c8f023f66aa0 req-b79750cd-f415-4ed5-a024-042b7cb8a92e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Synchronizing usage tracker for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:07:47.417027 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.7387090 May 25 16:07:47.436656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Request body: {'port': {'device_id': 'ee7483db-c622-4701-aa41-45aa235c8251', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:07:47.510306 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-6669f0c4-a8f2-4558-b610-dca0047193e5 None None] Provisioning for port db215e56-e99f-4ad2-a152-51e2aa8bb9ad completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:07:47.516172 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-6669f0c4-a8f2-4558-b610-dca0047193e5 None None] Provisioning complete for port db215e56-e99f-4ad2-a152-51e2aa8bb9ad triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:07:47.516662 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6669f0c4-a8f2-4558-b610-dca0047193e5 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:47.526126 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0d5a4a83-4af2-4443-97bf-c8f023f66aa0 req-b79750cd-f415-4ed5-a024-042b7cb8a92e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Unset dirty status for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:07:47.526920 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d5a4a83-4af2-4443-97bf-c8f023f66aa0 req-b79750cd-f415-4ed5-a024-042b7cb8a92e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 931 time: 0.3591464 May 25 16:07:47.538323 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e19ab7d9-dccb-417e-bbe1-b2f02edac37e req-3d561e4b-1b39-4749-af44-361f9172bf6d tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=cc5cddcb-4e4b-43a8-90e7-165b9b1045ac HTTP/1.1" status: 200 len: 210 time: 0.0717261 May 25 16:07:47.611589 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-6669f0c4-a8f2-4558-b610-dca0047193e5 None None] Port db215e56-e99f-4ad2-a152-51e2aa8bb9ad cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:07:47.649074 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d5a4a83-4af2-4443-97bf-c8f023f66aa0 req-06bea5d3-9c8a-48f5-b653-8b2b6c13b54e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1077332 May 25 16:07:47.654424 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:48.112804 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1ef9d883-e869-428f-a068-6e2b0c02e556 req-15103de5-b528-4fdc-a9fc-650928746e82 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:48.114355 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1ef9d883-e869-428f-a068-6e2b0c02e556 req-15103de5-b528-4fdc-a9fc-650928746e82 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.3182337 May 25 16:07:48.184467 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1ef9d883-e869-428f-a068-6e2b0c02e556 req-5fdfb25a-fa97-4100-ab48-c44199ed1fe9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0570703 May 25 16:07:48.265037 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id db215e56-e99f-4ad2-a152-51e2aa8bb9ad {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:07:48.275129 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:48.292688 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Transition to ACTIVE for port object db215e56-e99f-4ad2-a152-51e2aa8bb9ad will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:07:48.462215 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:48.467809 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:48.629636 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:48.657863 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.190s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:48.716252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Attempting to bind port db215e56-e99f-4ad2-a152-51e2aa8bb9ad on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:07:48.716656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Attempting to bind port db215e56-e99f-4ad2-a152-51e2aa8bb9ad by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:07:48.717055 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Attempting to bind port db215e56-e99f-4ad2-a152-51e2aa8bb9ad on network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:07:48.722741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Port db215e56-e99f-4ad2-a152-51e2aa8bb9ad on network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:07:48.723093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Attempting to bind port db215e56-e99f-4ad2-a152-51e2aa8bb9ad on network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:07:48.729148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 7, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:07:48.731303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Checking segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:07:48.731303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Bound using segment: {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:07:48.731570 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Bound port: db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '79c416e6-aabc-4638-95ed-e2e21a7b289f', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 920, 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:07:48.732259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:48.857430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:07:48.860773 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:07:48.947166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-edbc15db-f803-48f1-8fa4-f79923c4ae42 req-3a831955-0d4b-4053-a290-1a64d45fef40 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:48.948678 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-edbc15db-f803-48f1-8fa4-f79923c4ae42 req-3a831955-0d4b-4053-a290-1a64d45fef40 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0865688 May 25 16:07:48.950223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:48.954046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a83abf4c-409f-4eef-9d33-3134122f42b4 tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:49.035633 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.036230 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:49.037653 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] 10.222.0.42 "PUT /v2.0/ports/db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 1093 time: 1.6057243 May 25 16:07:49.038162 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:07:49.052868 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device f1cbd3dd-0666-4b74-80e3-36426f4ec54f no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:07:49.085818 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-edbc15db-f803-48f1-8fa4-f79923c4ae42 req-6cd35c0e-ccc6-4ac2-a404-69c4f9b2d848 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0994127 May 25 16:07:49.104525 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:49.108030 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.154s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:49.113893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: f1cbd3dd-0666-4b74-80e3-36426f4ec54f {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:07:49.115372 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device f1cbd3dd-0666-4b74-80e3-36426f4ec54f not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:07:49.132019 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'status': 'completed', 'code': 200} May 25 16:07:49.285934 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-0f696c10-1b04-4813-bc1f-de2cf048189f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.287755 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-0f696c10-1b04-4813-bc1f-de2cf048189f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1096 time: 0.0973239 May 25 16:07:49.508178 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-1739741d-f530-47b5-98ed-0d4cf3cbf166 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.509788 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-1739741d-f530-47b5-98ed-0d4cf3cbf166 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&shared=False HTTP/1.1" status: 200 len: 910 time: 0.2081373 May 25 16:07:49.522786 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-42ca1d3a-a488-4d6a-a529-2db03a005cb9 req-0ec536bc-e358-48e8-9bc8-6944a0e1e343 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.524274 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-42ca1d3a-a488-4d6a-a529-2db03a005cb9 req-0ec536bc-e358-48e8-9bc8-6944a0e1e343 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1967905 May 25 16:07:49.659456 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-42ca1d3a-a488-4d6a-a529-2db03a005cb9 req-14c534dc-4df5-4123-a854-3c89949b52a0 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1139152 May 25 16:07:49.694891 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-be330ea5-23ed-4998-9acc-be38c719f92e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.696357 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96088a0a-d1c2-47d0-ad17-50c3a4502fba req-be330ea5-23ed-4998-9acc-be38c719f92e service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1712251 May 25 16:07:49.837710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-bc5bd7fe-f7f4-47ed-8662-91798fced352 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:49.839183 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-bc5bd7fe-f7f4-47ed-8662-91798fced352 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1096 time: 0.0769660 May 25 16:07:50.292997 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0edfe8cd-fe14-4569-82e9-8a9461097bcb req-ba723a0e-5305-4590-8c82-fb1684ed498b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:50.295200 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0edfe8cd-fe14-4569-82e9-8a9461097bcb req-ba723a0e-5305-4590-8c82-fb1684ed498b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1007190 May 25 16:07:50.350453 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0edfe8cd-fe14-4569-82e9-8a9461097bcb req-83a584d0-78b9-4792-8e42-49cfc39a6d32 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0476437 May 25 16:07:50.504143 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46972) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:50.627090 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:50.641645 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:50.646250 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Usage tracker for resource:security_group_rule and tenant:8bea7151dc4f44edaf34258338923418 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:50.649306 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:50.655728 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:50.751256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:50.826440 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:50.835134 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-291f50a7-fbd8-4e78-8d60-55ca0fa1ada1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:50.880656 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=8bea7151dc4f44edaf34258338923418&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3754218 May 25 16:07:50.884158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 46980) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:50.889294 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6a0cc096-8f42-4c30-9c4b-912788193959,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:50.903040 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:50.910694 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e040448-a8b1-437a-ab53-757e8d67f7b2 req-cebfd6b9-b7c4-400d-84bb-2a3e2064f87b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:50.912283 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e040448-a8b1-437a-ab53-757e8d67f7b2 req-cebfd6b9-b7c4-400d-84bb-2a3e2064f87b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1371849 May 25 16:07:50.949407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:50.958122 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e040448-a8b1-437a-ab53-757e8d67f7b2 req-f1301209-a475-47fe-bce5-dfb8eec0b36a tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0399878 May 25 16:07:50.982771 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:51.145381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:51.149177 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-98c91f0c-3dde-4be1-8aea-f56bd3cf4084 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:51.160900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6a0cc096-8f42-4c30-9c4b-912788193959,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:51.166979 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:51.171975 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:51.171975 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/6a0cc096-8f42-4c30-9c4b-912788193959 HTTP/1.1" status: 204 len: 168 time: 0.2842619 May 25 16:07:51.657063 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6bad33b-1e78-4fee-a458-89b9a8754fc9 req-89dafd96-2ef3-434c-814d-9be0c779c35c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:51.658640 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6bad33b-1e78-4fee-a458-89b9a8754fc9 req-89dafd96-2ef3-434c-814d-9be0c779c35c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1970360 May 25 16:07:51.704873 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6bad33b-1e78-4fee-a458-89b9a8754fc9 req-7d6dda10-65a7-4276-a4ae-8addfbd2ec3b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0401809 May 25 16:07:51.879183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 46996) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:51.896604 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:51.911763 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:51.915589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Usage tracker for resource:security_group_rule and tenant:ea42b653206a41c3a9da9509bdf41489 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:51.918584 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:51.925264 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:52.020470 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.111822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.123105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-70f32a56-2a9f-4eba-9a30-fa58bbd779be admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:52.176049 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=e77fbf42-04f0-4442-928f-a6d04b34f113,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:52.179317 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:52.182574 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=ea42b653206a41c3a9da9509bdf41489&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3023746 May 25 16:07:52.185560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47002) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:52.207007 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8f987c7-1af9-4469-9675-7c77fdfc185b req-ca406e1a-a134-402e-b5fb-5c3656ed1fd8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:52.208641 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b8f987c7-1af9-4469-9675-7c77fdfc185b req-ca406e1a-a134-402e-b5fb-5c3656ed1fd8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1372118 May 25 16:07:52.253563 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.256005 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b8f987c7-1af9-4469-9675-7c77fdfc185b req-7aab0128-75a2-437b-a30d-0836444466f9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0413342 May 25 16:07:52.263373 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:07:52.278130 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.439073 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.442922 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8153cebe-f84a-495a-8b7e-a0ebadd0e784 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:52.456075 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=e77fbf42-04f0-4442-928f-a6d04b34f113,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:52.458982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:52.460891 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:52.462003 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/e77fbf42-04f0-4442-928f-a6d04b34f113 HTTP/1.1" status: 204 len: 168 time: 0.2746718 May 25 16:07:52.541338 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-e086d0cc-123d-49fa-bb18-e2870cc927cc service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 217 time: 0.0686479 May 25 16:07:52.639925 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-6fc51529-f8bc-4f8b-9c22-689a2ea56fdc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:52.641358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-6fc51529-f8bc-4f8b-9c22-689a2ea56fdc service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.0766671 May 25 16:07:52.733868 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e460302-e40d-4a78-947d-7511c9307520 req-aedccdd7-9b0f-4d64-b2d4-0f04d2f68453 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:52.735592 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-aedccdd7-9b0f-4d64-b2d4-0f04d2f68453 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0878937 May 25 16:07:52.943388 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-a283cd44-f123-42db-bcf4-b48b2260d5fc service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2004304 May 25 16:07:52.979804 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7f2c8963-d577-4444-bd55-9aa4a3f8d19b req-d80ead3d-b10b-4a1c-bf97-cf9b8c3f72e5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:52.981323 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f2c8963-d577-4444-bd55-9aa4a3f8d19b req-d80ead3d-b10b-4a1c-bf97-cf9b8c3f72e5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1585798 May 25 16:07:53.096714 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7f2c8963-d577-4444-bd55-9aa4a3f8d19b req-f9dc0627-6cab-4f80-8e76-b9f93201d034 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0669627 May 25 16:07:53.135309 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e460302-e40d-4a78-947d-7511c9307520 req-904c5280-66cd-4462-abe9-d0c98f8d12dc service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1847954 May 25 16:07:53.464243 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-95ca679f-eb99-4b55-a8fa-8aeeabaac200 req-dc27318c-7599-472e-b44b-5385527194be tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:53.473531 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95ca679f-eb99-4b55-a8fa-8aeeabaac200 req-dc27318c-7599-472e-b44b-5385527194be tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1065319 May 25 16:07:53.520803 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-95ca679f-eb99-4b55-a8fa-8aeeabaac200 req-24a16c98-756a-4e98-a18a-ce708b0a3bdd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0464022 May 25 16:07:54.175319 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cae8c95c-788d-4661-b92b-4f4e3ce89c27 req-28671ca1-98c7-4fac-a351-3cf1911b9aae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:54.177572 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cae8c95c-788d-4661-b92b-4f4e3ce89c27 req-28671ca1-98c7-4fac-a351-3cf1911b9aae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1426444 May 25 16:07:54.332237 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-cae8c95c-788d-4661-b92b-4f4e3ce89c27 req-3fbbdf75-a0e9-46b5-a303-383d5b841031 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1431098 May 25 16:07:54.384691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-b4974034-bc10-4a7d-a256-1d7b20739d14 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:54.386146 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-b4974034-bc10-4a7d-a256-1d7b20739d14 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 1098 time: 0.1651986 May 25 16:07:54.400150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b338f75a-764f-41ec-8f73-383a367a078b req-f375f0c8-31c7-4596-aca7-9f52ff3b12b5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:54.401724 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b338f75a-764f-41ec-8f73-383a367a078b req-f375f0c8-31c7-4596-aca7-9f52ff3b12b5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1215229 May 25 16:07:54.548703 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b338f75a-764f-41ec-8f73-383a367a078b req-ae465fe1-f0a1-4a91-93c0-f0206885827b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1256940 May 25 16:07:54.595892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-636d8c8d-c7d6-4465-926f-967b4dd6ddcb service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:54.597564 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-636d8c8d-c7d6-4465-926f-967b4dd6ddcb service neutron] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 912 time: 0.1948721 May 25 16:07:54.686327 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47070) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:54.710594 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-def63e3a-9fe1-4b2c-abb2-bd6dc9d52184 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 200 len: 1001 time: 0.1052239 May 25 16:07:54.781821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-f5752656-a664-4c17-ac9e-319742da61a9 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:54.784959 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-f5752656-a664-4c17-ac9e-319742da61a9 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.0667400 May 25 16:07:54.824904 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:54.846280 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:07:54.852340 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Usage tracker for resource:security_group_rule and tenant:92a55e416ead4a44b15a07e51d916fdf is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:07:54.856609 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:07:54.865654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:07:55.005055 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-aa4b4139-dd0f-4ba7-b4de-5f68c2d5928d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:55.006695 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-aa4b4139-dd0f-4ba7-b4de-5f68c2d5928d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.2156754 May 25 16:07:55.014940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.084714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.092535 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-372ab88e-b9fe-4e1b-b594-6342fa20adc4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:55.145842 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=92a55e416ead4a44b15a07e51d916fdf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4578102 May 25 16:07:55.147451 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=98f9cb4b-96a0-42a2-80b8-ed0c95b11305,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:55.151953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47088) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:07:55.152467 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:55.180794 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-7d4d7ab0-9e5d-4837-855b-72fd44ee10e5 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1652625 May 25 16:07:55.310442 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.364882 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.387854 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7a58c39f-5621-40e6-8ed5-6ebc962cd5c3 req-1091b8e0-54bb-413a-b6dc-cc0f80093241 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1971376 May 25 16:07:55.534449 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1094f07a-7fab-42a4-a02d-a9f4399049b6 req-f8d069b7-0230-4f9b-87e9-e0fe65a7d586 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:55.535988 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1094f07a-7fab-42a4-a02d-a9f4399049b6 req-f8d069b7-0230-4f9b-87e9-e0fe65a7d586 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0884938 May 25 16:07:55.537128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.540817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e43a0f8e-a612-48ae-8aa9-d0f78c0ec968 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:07:55.569050 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=98f9cb4b-96a0-42a2-80b8-ed0c95b11305,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:07:55.575745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:07:55.603353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:07:55.603353 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/98f9cb4b-96a0-42a2-80b8-ed0c95b11305 HTTP/1.1" status: 204 len: 168 time: 0.4485955 May 25 16:07:55.607970 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1094f07a-7fab-42a4-a02d-a9f4399049b6 req-cc3e87db-e348-414b-81e2-316581fd0d81 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0575428 May 25 16:07:55.771979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc6c14c4-f4b0-44f3-82e7-5522e887ed43 req-94d89480-3c4f-4688-bbcc-b79243cc2746 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:55.773466 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc6c14c4-f4b0-44f3-82e7-5522e887ed43 req-94d89480-3c4f-4688-bbcc-b79243cc2746 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0887892 May 25 16:07:55.819997 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc6c14c4-f4b0-44f3-82e7-5522e887ed43 req-0e851a17-34f7-4700-b56e-d05f3366d23d tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0403576 May 25 16:07:56.808636 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9ddc3172-0af2-4ca2-a789-de5879520d7e req-2e2473ae-a74c-412b-806b-b645d984fbe6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:56.810280 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9ddc3172-0af2-4ca2-a789-de5879520d7e req-2e2473ae-a74c-412b-806b-b645d984fbe6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0899282 May 25 16:07:56.862114 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9ddc3172-0af2-4ca2-a789-de5879520d7e req-5f9e87ed-9b2f-4ed8-a1aa-3305962c18fe tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0450351 May 25 16:07:57.031133 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f6586771-d501-4e23-8058-73b3e2e5603e req-beba42f4-7196-45e9-ba44-6307c7bea094 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:57.032661 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f6586771-d501-4e23-8058-73b3e2e5603e req-beba42f4-7196-45e9-ba44-6307c7bea094 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0908532 May 25 16:07:57.084530 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f6586771-d501-4e23-8058-73b3e2e5603e req-12665e41-2ee3-4849-b47d-de31f04f0212 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0459409 May 25 16:07:58.282162 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5644b213-e4ee-4bf9-80c6-f66422de273b req-c4c1c273-31af-4283-a494-9929cd511da1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:58.284219 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5644b213-e4ee-4bf9-80c6-f66422de273b req-c4c1c273-31af-4283-a494-9929cd511da1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0860536 May 25 16:07:58.333398 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5644b213-e4ee-4bf9-80c6-f66422de273b req-f82a9baa-6734-4dba-bcec-253fe914bb52 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423234 May 25 16:07:59.534533 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-65e8fcb8-5c91-4fdf-bec8-453697f92651 req-9cd2eea9-022f-40d7-8466-7fd21fef3f3a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:07:59.536012 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-65e8fcb8-5c91-4fdf-bec8-453697f92651 req-9cd2eea9-022f-40d7-8466-7fd21fef3f3a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0896990 May 25 16:07:59.591694 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-65e8fcb8-5c91-4fdf-bec8-453697f92651 req-1600eccb-deca-4c27-8a72-93c7819c6a17 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0493677 May 25 16:08:00.433414 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-8c61b134-6e6a-445c-9960-8532ac157b95 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:00.434906 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-8c61b134-6e6a-445c-9960-8532ac157b95 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/networks?id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f HTTP/1.1" status: 200 len: 813 time: 0.1558385 May 25 16:08:00.450029 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-d963f2e0-e328-4a9e-a277-8eee3f817dcc tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/quotas/e8bb2885c2ca420b852158159d30e3b4 HTTP/1.1" status: 200 len: 375 time: 0.0093253 May 25 16:08:00.500117 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-2c6580e3-4b65-44e3-9f11-8fb0171de00c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&fields=id HTTP/1.1" status: 200 len: 210 time: 0.0438941 May 25 16:08:00.648149 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-57391b60-cb53-4453-bf6f-0dfbd551c637 service neutron] 10.222.0.82 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1406629 May 25 16:08:00.852148 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-5771dc69-b9a3-4fd4-b6bb-9334496c25be service neutron] 10.222.0.82 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1925387 May 25 16:08:00.866681 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db6d8bc6-06da-4c8c-affe-dfacf1c0f2c9 req-86c579b4-7dce-4bc6-aed8-60bc55d2bc5b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:00.868152 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db6d8bc6-06da-4c8c-affe-dfacf1c0f2c9 req-86c579b4-7dce-4bc6-aed8-60bc55d2bc5b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1339521 May 25 16:08:00.913656 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db6d8bc6-06da-4c8c-affe-dfacf1c0f2c9 req-b9d5fd04-a549-42dd-8f01-29b9daf0118e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0396335 May 25 16:08:01.129368 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:08:01.182679 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7b48cf60-06ba-463f-b6b8-fd40ea125b15 req-376662f1-99df-4781-ae14-8abcd1e3d07b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0525904 May 25 16:08:01.330031 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:01.345690 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'mac_address': 'fa:16:3e:2b:e0:91', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 920, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:08:01.405322 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:01.422831 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id db215e56-e99f-4ad2-a152-51e2aa8bb9ad {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:08:01.442723 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:01.597039 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:01.600577 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:01.687527 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:01.695981 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.696459 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.696800 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.697130 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.697449 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.697770 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.698155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.698474 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.698786 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.699126 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.699440 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.699749 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:01.705570 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.105s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:01.909051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['db215e56-e99f-4ad2-a152-51e2aa8bb9ad'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:08:02.224560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f09895c4-8b1f-47f4-ad4d-5dbfb83651b3 req-1fea0caa-00a5-403b-a069-c8c5fdfd1117 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:02.226100 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f09895c4-8b1f-47f4-ad4d-5dbfb83651b3 req-1fea0caa-00a5-403b-a069-c8c5fdfd1117 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 944 time: 0.1958528 May 25 16:08:02.277444 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f09895c4-8b1f-47f4-ad4d-5dbfb83651b3 req-f6ac9212-b912-4eb7-97a7-e3ce35dcd9ae tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0444531 May 25 16:08:02.371143 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4faa5484-836a-4716-b795-ec16b7a130a0 req-fd92f3b0-9b27-4bfb-bd47-2bcc77c38487 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0429125 May 25 16:08:03.761879 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-269a8089-f1b3-4ff7-ad17-2c39232483aa req-08aa05c0-c056-4cb7-bdd6-67b836969cef tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.2019486 May 25 16:08:03.820932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74140bad-f0f6-4e8d-a022-aac2b4e5d753 req-75b98ed8-f62e-483f-a167-002763d75886 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:03.828217 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74140bad-f0f6-4e8d-a022-aac2b4e5d753 req-75b98ed8-f62e-483f-a167-002763d75886 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 944 time: 0.4175990 May 25 16:08:03.876203 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74140bad-f0f6-4e8d-a022-aac2b4e5d753 req-1278bf5f-cf50-403b-972e-a8ca359304b1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0478082 May 25 16:08:04.666283 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:08:04.818673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port db215e56-e99f-4ad2-a152-51e2aa8bb9ad completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:08:04.826797 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port db215e56-e99f-4ad2-a152-51e2aa8bb9ad triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:08:04.826797 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:04.955295 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:05.071584 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-42e02614-cf40-4409-a554-b5ae9341b1c2 req-e2e458c7-f13c-4ab2-a5fe-657804fc2f6b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0519652 May 25 16:08:05.073707 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:05.073707 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:05.079667 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'code': 200} May 25 16:08:05.127823 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e5aa3e80-aa5b-47c7-8acb-3e86e21f04c1 req-1f158408-11bf-4797-ae03-01eb1af61682 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:05.129422 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e5aa3e80-aa5b-47c7-8acb-3e86e21f04c1 req-1f158408-11bf-4797-ae03-01eb1af61682 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 944 time: 0.1155441 May 25 16:08:05.217889 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e5aa3e80-aa5b-47c7-8acb-3e86e21f04c1 req-4ba615d5-c75c-48a2-aa47-0f64215addb9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0804703 May 25 16:08:05.261956 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:05.264781 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:05.335276 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:05.338429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:05.644185 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-721bec08-7a82-477e-a2a3-b3fbacb62542 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:06.245316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-13aa9215-27a0-44ce-aa63-92bebd3806bb req-edb0a786-486f-4899-b098-5fd23f5d747f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0586677 May 25 16:08:06.277734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f62981ee-3e4a-4a5a-8a5c-d023c76fc8d8 req-0ce8a290-5813-4a93-bbaf-e6e3968ddeaf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:06.278293 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f62981ee-3e4a-4a5a-8a5c-d023c76fc8d8 req-0ce8a290-5813-4a93-bbaf-e6e3968ddeaf service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.1863937 May 25 16:08:06.399847 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-aaca6937-940a-48a0-add1-ed9bd54c5437 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:06.401406 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-aaca6937-940a-48a0-add1-ed9bd54c5437 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "GET /v2.0/networks?id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f HTTP/1.1" status: 200 len: 813 time: 0.1637852 May 25 16:08:06.424466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-544a73aa-3713-42e4-b2bc-b04b9375f55c req-d8011660-dcde-4499-9375-196a6cb79bb1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:06.426435 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-544a73aa-3713-42e4-b2bc-b04b9375f55c req-d8011660-dcde-4499-9375-196a6cb79bb1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0924084 May 25 16:08:06.479434 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-544a73aa-3713-42e4-b2bc-b04b9375f55c req-5129f099-2594-472e-9c97-2867a114a828 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0468884 May 25 16:08:07.441498 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-571a76ae-71dc-4463-bdd7-0a35a0c3a9e0 req-e3a39017-73f7-42b3-b24b-7604bac953b4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0445440 May 25 16:08:07.669150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-30d696b4-c018-489a-9219-52fbaa33ae9c req-d7549fc7-49b3-4d72-9988-c9b25211c0db tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:07.670660 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-30d696b4-c018-489a-9219-52fbaa33ae9c req-d7549fc7-49b3-4d72-9988-c9b25211c0db tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0822103 May 25 16:08:07.725130 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-30d696b4-c018-489a-9219-52fbaa33ae9c req-efdb5a7d-3a2d-4358-923f-ab42769e5be9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0474415 May 25 16:08:08.155379 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:08.441152 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c14a031d-f457-4ea2-ac65-48681ef10860 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:08.739241 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ec8831a-6534-4808-9fcd-fedf8a1b8713 req-f48c4847-5e88-4bb4-adbd-235b00efb27a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0436678 May 25 16:08:08.913209 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5dbd63c-c9e9-4e3a-b62d-56d9c5f59d56 req-ab7aa02b-3f88-4b6e-b372-26d00365e751 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:08.914658 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5dbd63c-c9e9-4e3a-b62d-56d9c5f59d56 req-ab7aa02b-3f88-4b6e-b372-26d00365e751 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0793064 May 25 16:08:08.962420 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5dbd63c-c9e9-4e3a-b62d-56d9c5f59d56 req-f13148d7-b461-4b00-a37d-b4d9f0bf56e2 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0420802 May 25 16:08:09.084162 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:09.102819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Request body: {'port': {'device_id': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f', 'admin_state_up': True, 'tenant_id': 'e8bb2885c2ca420b852158159d30e3b4'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:09.105399 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f62981ee-3e4a-4a5a-8a5c-d023c76fc8d8 req-9f670ca3-480d-4192-860c-ea1edcd14c66 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0361974 May 25 16:08:09.333993 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:09.339361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Usage tracker for resource:port and tenant:e8bb2885c2ca420b852158159d30e3b4 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:09.342538 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Quota usage for port was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:09.348999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attempting to reserve 1 items for resource port. Total usage: 0; quota limit: 500; headroom:500 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:09.484362 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Made reservation on behalf of e8bb2885c2ca420b852158159d30e3b4 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:08:09.488208 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:09.663433 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Allocated IP 172.20.0.9 (610dbdeb-d88d-45f7-b872-9563bbbe7c3f/2521d219-c255-41fe-80ad-cbccc4e9c1fe/2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:08:09.872052 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:09.897196 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5b91d84-ffdb-4d2e-b1d2-5e6cf5bfae99 req-5d713adb-7e0f-45b9-8f33-50a0bb4bad11 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0446067 May 25 16:08:09.962043 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:09.968409 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-281241bc-cda0-4882-b103-b66f0a306205 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:10.038380 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:08:10.041540 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:08:10.046256 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:08:10.082300 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Pushing event updated for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:10.091627 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:10.115303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:10.116104 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:10.155927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8c7095ad-aa4b-4b05-b07f-8f54714b6fad req-731123a3-a20f-4b77-84b2-54c2c51f67b5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:10.157268 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8c7095ad-aa4b-4b05-b07f-8f54714b6fad req-731123a3-a20f-4b77-84b2-54c2c51f67b5 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0874770 May 25 16:08:10.208309 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8c7095ad-aa4b-4b05-b07f-8f54714b6fad req-78085470-c01f-422c-9187-9d02d13030e3 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0442102 May 25 16:08:10.219053 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1203454 May 25 16:08:10.297794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Request body: {'port': {'device_id': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:10.414424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:10.453083 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:10.795487 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:08:10.811803 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:10.830903 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Transition to ACTIVE for port object 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:08:10.944399 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:10.948857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bac4110d-3835-45a5-82df-fda085ec50bd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:11.108166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9bbdfc27-f719-4447-a4b6-50376dcb35f8 req-fc4050eb-8069-4e20-b597-b3a3c2fa1056 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.109481 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9bbdfc27-f719-4447-a4b6-50376dcb35f8 req-fc4050eb-8069-4e20-b597-b3a3c2fa1056 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 943 time: 0.0840347 May 25 16:08:11.116717 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Pushing event updated for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:11.127138 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.178s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:11.155237 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:11.155803 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9bbdfc27-f719-4447-a4b6-50376dcb35f8 req-fde12b68-9b68-4427-b22a-f7fc5b21771e tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403631 May 25 16:08:11.196108 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Attempting to bind port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:08:11.196446 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Attempting to bind port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:08:11.196828 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Attempting to bind port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:08:11.201588 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:08:11.201902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Attempting to bind port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 on network 610dbdeb-d88d-45f7-b872-9563bbbe7c3f {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:08:11.207626 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 8, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:08:11.211780 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Checking segment: {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:08:11.211780 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Bound using segment: {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:08:11.211780 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Bound port: 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'f1cda128-299e-4975-835f-ff55ee624d0a', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 947, 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:08:11.211780 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:11.310813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:08:11.314660 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:08:11.412438 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9faad73b-9b54-4fd3-ab59-8576162d7e8c req-5a38df30-9a9f-4542-8ed0-aa5e0f4eeb50 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.413911 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9faad73b-9b54-4fd3-ab59-8576162d7e8c req-5a38df30-9a9f-4542-8ed0-aa5e0f4eeb50 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0942163 May 25 16:08:11.460089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9faad73b-9b54-4fd3-ab59-8576162d7e8c req-0dcf669f-53d9-42dd-a60e-910c596990d6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0397372 May 25 16:08:11.527576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:11.533073 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fbeae190-589e-4e5f-8696-c3681ebbe217 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:11.605069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.605759 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:11.607248 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] 10.222.0.42 "PUT /v2.0/ports/2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 HTTP/1.1" status: 200 len: 1093 time: 1.3139682 May 25 16:08:11.607813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:11.634581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Pushing event updated for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:11.640868 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:11.672095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a8f8f0b-7369-46bb-ada8-443f1c80ab23 req-406b8558-a747-4c90-805d-4919cad88810 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.674635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9a8f8f0b-7369-46bb-ada8-443f1c80ab23 req-406b8558-a747-4c90-805d-4919cad88810 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1020865 May 25 16:08:11.681061 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44', 'status': 'completed', 'code': 200} May 25 16:08:11.726994 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9a8f8f0b-7369-46bb-ada8-443f1c80ab23 req-a857f1af-c190-4edf-8f60-e5b1e845dd8c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0444396 May 25 16:08:11.757842 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47380) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:11.808817 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-4d0e7d3c-3453-4ba2-a15b-ff667bf3ff27 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.810360 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-4d0e7d3c-3453-4ba2-a15b-ff667bf3ff27 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 1096 time: 0.0736251 May 25 16:08:11.982222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-f0af73e5-f467-41af-b730-748398cedd1b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:11.984007 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-f0af73e5-f467-41af-b730-748398cedd1b service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=e8bb2885c2ca420b852158159d30e3b4&shared=False HTTP/1.1" status: 200 len: 914 time: 0.1515453 May 25 16:08:12.030462 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-f656f5fe-0b1a-44d0-8a2e-8a1ff431cbcf service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.1144702 May 25 16:08:12.036737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-f3566de3-cad5-4ee1-94c2-9c5eb9f30606 tempest-TestShelveInstance-431247030 tempest-TestShelveInstance-431247030-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.039549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f3566de3-cad5-4ee1-94c2-9c5eb9f30606 tempest-TestShelveInstance-431247030 tempest-TestShelveInstance-431247030-project-admin] Synchronizing usage tracker for tenant:649773106ce5438aaa40fb240586ad11 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:12.166560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f3566de3-cad5-4ee1-94c2-9c5eb9f30606 tempest-TestShelveInstance-431247030 tempest-TestShelveInstance-431247030-project-admin] Unset dirty status for tenant:649773106ce5438aaa40fb240586ad11 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:12.171727 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f3566de3-cad5-4ee1-94c2-9c5eb9f30606 tempest-TestShelveInstance-431247030 tempest-TestShelveInstance-431247030-project-admin] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1093 time: 0.4095395 May 25 16:08:12.176988 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47386) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:12.188966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'fixed_ip_address': '172.20.0.7'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:12.195229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:12.212522 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Usage tracker for resource:floatingip and tenant:5d4b9c0f27af44a9ace8862a2133a0cb is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:12.217996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:12.227626 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:12.231531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-0e67a0d1-f2ca-45cf-9abd-1d7c086735eb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.233198 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-0e67a0d1-f2ca-45cf-9abd-1d7c086735eb service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 1100 time: 0.1080680 May 25 16:08:12.284560 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-5a40f0c0-bd79-45b7-857f-06b85a113351 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.286090 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-617db406-f0c6-4f86-8ced-8f234dd7e428 req-5a40f0c0-bd79-45b7-857f-06b85a113351 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2917609 May 25 16:08:12.421842 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for floatingip, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:12.546611 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-da135d42-6591-4834-b2c3-872659c86065 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.548226 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-da135d42-6591-4834-b2c3-872659c86065 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=e8bb2885c2ca420b852158159d30e3b4&device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 1096 time: 0.1802621 May 25 16:08:12.552089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-1759a882-2989-4506-9ba3-46495bcbd4d5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.553779 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-1759a882-2989-4506-9ba3-46495bcbd4d5 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.2989631 May 25 16:08:12.570014 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-88d1087e-7364-44e4-a9df-fa074caaa9af req-c2234b33-7004-45bf-ad1c-288c6575d306 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.571699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-88d1087e-7364-44e4-a9df-fa074caaa9af req-c2234b33-7004-45bf-ad1c-288c6575d306 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 943 time: 0.2461636 May 25 16:08:12.644347 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-4fb02444-e7e2-4efd-9796-e77f94c1f828 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.9&port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 HTTP/1.1" status: 200 len: 217 time: 0.0758266 May 25 16:08:12.706193 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-88d1087e-7364-44e4-a9df-fa074caaa9af req-dc8fb9e4-9f09-4e06-b2cf-fbaba0d0cfeb tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0727913 May 25 16:08:12.715075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-20d4d9ba-3ec8-4731-86b8-d564bf2b1045 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.12&port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 200 len: 1002 time: 0.1420560 May 25 16:08:12.730197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:12.733468 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-5691275e-b465-4548-8b2c-b07194aab469 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.735049 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-5691275e-b465-4548-8b2c-b07194aab469 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2521d219-c255-41fe-80ad-cbccc4e9c1fe HTTP/1.1" status: 200 len: 853 time: 0.0772760 May 25 16:08:12.800122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-d7973fa0-da80-460d-933c-d1030d6c3c81 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.802716 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-d7973fa0-da80-460d-933c-d1030d6c3c81 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0806453 May 25 16:08:12.821756 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-56fe32c5-5dc7-426a-92e1-08bf44b06ea1 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=610dbdeb-d88d-45f7-b872-9563bbbe7c3f&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 210 time: 0.0479150 May 25 16:08:12.882747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-eaed1972-7107-43c2-a6d1-6a35571129bf service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:12.884281 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-eaed1972-7107-43c2-a6d1-6a35571129bf service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0749655 May 25 16:08:13.033895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Allocated IP 172.27.2.99 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/1d6fc6fc-6fb4-4c4a-b162-708178931b62) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:08:13.040236 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-90a7c124-f870-42bb-ae13-31eab087f795 service neutron] 10.222.0.42 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2043805 May 25 16:08:13.040429 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-8bf4c409-a746-4e11-ac27-f4ee24b1c585 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1365635 May 25 16:08:13.190672 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57c82c19-d0d9-45ac-8cea-3f00d8311e16 req-07c8c136-c76c-44f8-a276-20cd00880b6d service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1379983 May 25 16:08:13.203076 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47406) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:13.300334 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-d0d9b455-2ac0-469e-8a41-bcbed00ef4f0 req-393555fb-b1b7-4617-bef1-9a1d26da19be service neutron] 10.222.0.42 "GET /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2362037 May 25 16:08:13.308357 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.334403 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.350098 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:13.355381 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Usage tracker for resource:security_group_rule and tenant:c9b5ba641913415795c44db1d4417ab9 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:13.358589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:13.365939 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:13.455027 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.586761 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.587352 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.591637 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-df5a4e7e-3bf4-4500-88e5-686b6b6c18b5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:13.596915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-20d139e2-6cbf-4333-a51c-b42287c83648 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:13.636834 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=c9b5ba641913415795c44db1d4417ab9&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4323301 May 25 16:08:13.638601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9bd8f5d5-57b5-4d6c-9551-b303a9281b10,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:13.641870 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:13.642513 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47412) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:13.660296 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:08:13.663184 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:08:13.666767 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:08:13.696411 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'Port': ['ID=1d6fc6fc-6fb4-4c4a-b162-708178931b62,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:13.702538 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:13.705038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.734270 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.895832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.900943 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7aa475dc-c442-45c0-9ef3-d6621e7fdaf6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:13.918708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9bd8f5d5-57b5-4d6c-9551-b303a9281b10,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:13.921197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:13.932608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:13.933780 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/9bd8f5d5-57b5-4d6c-9551-b303a9281b10 HTTP/1.1" status: 204 len: 168 time: 0.2902968 May 25 16:08:13.975604 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4f445665-b72e-40e6-a6b6-611815edcdfd req-4417c95d-1359-4b49-b217-a6a9f6e39bae tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:13.977253 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f445665-b72e-40e6-a6b6-611815edcdfd req-4417c95d-1359-4b49-b217-a6a9f6e39bae tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 943 time: 0.1399767 May 25 16:08:14.039115 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4f445665-b72e-40e6-a6b6-611815edcdfd req-05941b6b-2de7-4b4c-b46f-cba53906ba75 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0553000 May 25 16:08:14.265622 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.467466 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.754375 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.820964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47440) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:14.838183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.852633 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:14.856342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Usage tracker for resource:security_group_rule and tenant:54deb50c797745a99528f22c4c4e30cf is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:14.859105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:14.863195 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.865681 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:14.868300 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:14.954430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:14.983476 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event updated for resources: {'Port': ['ID=1d6fc6fc-6fb4-4c4a-b162-708178931b62,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:14.994755 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.995144 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.995497 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.995847 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.996190 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.996533 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.997014 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.997343 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.997684 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.998021 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.998356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:14.998722 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:15.002557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:15.006015 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.044427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.052944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d7fb267e-cadd-4ff6-aef1-76bddf4af365 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:15.098392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c87a89a7-c058-456f-ba0e-5fc2a82abba9,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:15.102764 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=54deb50c797745a99528f22c4c4e30cf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2807693 May 25 16:08:15.103775 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:15.105263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47446) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:15.179417 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:08:15.214379 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Floating IP a7837106-14c5-4cd8-9867-49d40c2bd9c1 associated. External IP: 172.27.2.99, port: db215e56-e99f-4ad2-a152-51e2aa8bb9ad. May 25 16:08:15.232289 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.271245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.316873 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf5ab240-f072-4f9e-a85c-0ccfe49ee9b1 req-3de33d28-84b7-4b07-a397-648644d7fd9c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:15.318358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf5ab240-f072-4f9e-a85c-0ccfe49ee9b1 req-3de33d28-84b7-4b07-a397-648644d7fd9c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 943 time: 0.1528931 May 25 16:08:15.330213 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:15.346354 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44', 'network_id': '610dbdeb-d88d-45f7-b872-9563bbbe7c3f', 'port_id': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44', 'mac_address': 'fa:16:3e:a6:0e:41', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 947, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2521d219-c255-41fe-80ad-cbccc4e9c1fe', 'ip_address': '172.20.0.9'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:08:15.379571 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf5ab240-f072-4f9e-a85c-0ccfe49ee9b1 req-f365f490-a841-47e3-8645-5c5b99af9f1e tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0504169 May 25 16:08:15.413712 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.431619 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:08:15.448257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:15.523787 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.527733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:15.539807 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c87a89a7-c058-456f-ba0e-5fc2a82abba9,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:15.543573 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:15.547150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.550781 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/c87a89a7-c058-456f-ba0e-5fc2a82abba9 HTTP/1.1" status: 204 len: 168 time: 0.4420543 May 25 16:08:15.592518 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.594454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:15.604146 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Router ac04f176-2f32-43dd-b873-426ed50b356d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:08:15.667896 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:15.671867 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:15.814411 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:08:15.827329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Sync routers for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:08:15.840551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:08:15.843155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:15.930840 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:15.955274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:08:16.034724 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 3.8558688 May 25 16:08:16.037494 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:16.114074 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'status': 'completed', 'code': 200} May 25 16:08:16.216163 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:08:16.254786 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-38240f2b-7d23-4ffc-aa9b-1d786432ac72 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:16.256333 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-38240f2b-7d23-4ffc-aa9b-1d786432ac72 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.0913868 May 25 16:08:16.518809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47466) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:16.551502 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:16.564916 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-578fa96b-6d28-4c1f-88bb-20bd330ba73b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:16.566687 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-578fa96b-6d28-4c1f-88bb-20bd330ba73b service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.2973831 May 25 16:08:16.571731 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:16.577203 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Usage tracker for resource:security_group_rule and tenant:5be526033b1f48c190f1f8289c2af8af is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:16.590802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d1233e9e-c6c0-4b7f-aa40-237dfe7c989f req-a0283e52-6cc2-40b5-b018-0eef39158018 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:16.592464 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1233e9e-c6c0-4b7f-aa40-237dfe7c989f req-a0283e52-6cc2-40b5-b018-0eef39158018 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 944 time: 0.0906661 May 25 16:08:16.643746 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d1233e9e-c6c0-4b7f-aa40-237dfe7c989f req-19b8fe6c-30d6-4253-9982-6b814de2510e tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0444627 May 25 16:08:16.645274 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:16.653724 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'a7837106-14c5-4cd8-9867-49d40c2bd9c1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'floating_ip_address': '172.27.2.99', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 438, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'mac_address': 'fa:16:3e:2b:e0:91', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ee7483db-c622-4701-aa41-45aa235c8251', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:14Z', 'updated_at': '2021-05-25T16:08:14Z', 'revision_number': 0, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:16.658407 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Checking router: ac04f176-2f32-43dd-b873-426ed50b356d for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:08:16.667005 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:08:16.667615 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:16.672256 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:16.708212 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-b7317d2e-52c8-4c32-a314-267465977302 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 998 time: 0.1322994 May 25 16:08:16.798943 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-b1c4c453-414e-4a6a-8e42-58e9ef72d9df service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:16.800290 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-b1c4c453-414e-4a6a-8e42-58e9ef72d9df service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.0844746 May 25 16:08:16.859569 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:08:16.880547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:16.900944 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-d0cec196-1dc0-4bda-8603-ba344f3c9df3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:16.902445 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-d0cec196-1dc0-4bda-8603-ba344f3c9df3 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0946476 May 25 16:08:16.987898 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.003133 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3c3fdd87-0132-461c-a122-1fd5c07e6ead admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:17.054353 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2a2da44c-48fe-46c1-ba47-98e969d5c2dc,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:17.068675 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=5be526033b1f48c190f1f8289c2af8af&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5482795 May 25 16:08:17.069387 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:17.071329 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47478) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:17.100552 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-ecee1e2e-3a30-4b23-85da-3a4af12f8fce service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1906006 May 25 16:08:17.148233 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.184780 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:08:17.189159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.278606 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-bff8b940-d75e-43a4-959e-25fe332d2486 req-6469a673-4d62-4cdb-aa8f-7187b22fe890 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1692924 May 25 16:08:17.346430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'a7837106-14c5-4cd8-9867-49d40c2bd9c1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'floating_ip_address': '172.27.2.99', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 438, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'mac_address': 'fa:16:3e:2b:e0:91', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ee7483db-c622-4701-aa41-45aa235c8251', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:14Z', 'updated_at': '2021-05-25T16:08:14Z', 'revision_number': 0, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:17.351464 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bc9f3b5e-1741-46bf-9ead-455e80af2bc0 None None] The sync data for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi is: [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'a7837106-14c5-4cd8-9867-49d40c2bd9c1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'floating_ip_address': '172.27.2.99', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'fixed_ip_address': '172.20.0.7', 'status': 'DOWN', 'standard_attr_id': 438, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'mac_address': 'fa:16:3e:2b:e0:91', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ee7483db-c622-4701-aa41-45aa235c8251', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:14Z', 'updated_at': '2021-05-25T16:08:14Z', 'revision_number': 0, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:08:17.390346 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.396226 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-835cf806-20b5-4e17-a9b6-a986493ea190 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:17.418172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2a2da44c-48fe-46c1-ba47-98e969d5c2dc,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:17.421550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.025s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:17.423862 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.428221 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2a2da44c-48fe-46c1-ba47-98e969d5c2dc HTTP/1.1" status: 204 len: 168 time: 0.3520091 May 25 16:08:17.533017 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:08:17.548450 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:08:17.550270 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.686755 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-39eca830-d52c-43dc-bef1-d815a382b0ff None None] New status for floating IP a7837106-14c5-4cd8-9867-49d40c2bd9c1: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:08:17.795109 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.833164 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:17.844962 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:17.907593 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:17.910764 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:17.942994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-78dee1f6-6bc9-4f27-8408-7b26d7210ffd req-9b4eb533-97c4-4c76-8cf3-7de3a2cca3cf tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:17.968069 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-78dee1f6-6bc9-4f27-8408-7b26d7210ffd req-9b4eb533-97c4-4c76-8cf3-7de3a2cca3cf tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 944 time: 0.1573148 May 25 16:08:18.054047 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44', 'code': 200} May 25 16:08:18.096001 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:18.103869 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:18.121245 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-78dee1f6-6bc9-4f27-8408-7b26d7210ffd req-d160f20e-246f-44f9-a0af-aaa8f710a55c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1401072 May 25 16:08:18.662557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e82d15ab-b0f9-4a87-bb98-86ace371bbd1 req-5bdf384a-a2ac-40ae-8de2-eea7ff7173b4 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:18.664074 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e82d15ab-b0f9-4a87-bb98-86ace371bbd1 req-5bdf384a-a2ac-40ae-8de2-eea7ff7173b4 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 1098 time: 0.1930137 May 25 16:08:18.710365 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e82d15ab-b0f9-4a87-bb98-86ace371bbd1 req-b8f3df58-a64b-429b-96ca-79624f0efb68 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0368507 May 25 16:08:19.327640 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f87a3b5f-a00d-4ac5-b593-dbe326040015 req-d478ed83-85f5-4491-911f-40e8ffd3f59c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:19.329304 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f87a3b5f-a00d-4ac5-b593-dbe326040015 req-d478ed83-85f5-4491-911f-40e8ffd3f59c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0907435 May 25 16:08:19.385512 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f87a3b5f-a00d-4ac5-b593-dbe326040015 req-ed96f427-3e84-43a4-8d8a-2941409f0f81 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0494354 May 25 16:08:20.592271 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d2fc06d-9866-4e2a-8d86-6d89115a2cb3 req-275a363a-7e9a-4f11-9ee7-c5e1548bd4bd tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:20.593899 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9d2fc06d-9866-4e2a-8d86-6d89115a2cb3 req-275a363a-7e9a-4f11-9ee7-c5e1548bd4bd tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0842583 May 25 16:08:20.644620 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9d2fc06d-9866-4e2a-8d86-6d89115a2cb3 req-6e476b95-44ab-4ca9-974e-545de1ac237c tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0449038 May 25 16:08:22.028963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aeca1869-d3f1-4ff6-a9ae-07b8c2261459 req-89433eb4-ae4d-4e93-a0e4-9afb8e312140 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:22.030316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aeca1869-d3f1-4ff6-a9ae-07b8c2261459 req-89433eb4-ae4d-4e93-a0e4-9afb8e312140 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0820384 May 25 16:08:22.074952 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aeca1869-d3f1-4ff6-a9ae-07b8c2261459 req-86bd758c-4209-4304-9dbb-50a9ef0cd09a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0386598 May 25 16:08:23.363303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1c1f44d5-d09a-4b50-b98c-8104337878f3 req-e112d130-bfe2-4820-be74-ec7892fb9319 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:23.364728 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1c1f44d5-d09a-4b50-b98c-8104337878f3 req-e112d130-bfe2-4820-be74-ec7892fb9319 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.1688735 May 25 16:08:23.412857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1c1f44d5-d09a-4b50-b98c-8104337878f3 req-8310031b-1f54-4dcf-a793-b97ba0c0e433 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0420399 May 25 16:08:24.614637 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20094175-d836-4e14-9d80-a7d5db0d6730 req-ff61bdd7-1739-46ae-b5cd-c02be3586b7e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:24.616179 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20094175-d836-4e14-9d80-a7d5db0d6730 req-ff61bdd7-1739-46ae-b5cd-c02be3586b7e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0918820 May 25 16:08:24.681615 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20094175-d836-4e14-9d80-a7d5db0d6730 req-2c4380e2-fa96-467a-94d2-1677c2328a64 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0594063 May 25 16:08:25.879137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74e2c317-925a-40a6-9422-b3ab62b1828d req-ba7ecbb5-f384-44f5-bc5b-2d5b8460bef1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:25.880633 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74e2c317-925a-40a6-9422-b3ab62b1828d req-ba7ecbb5-f384-44f5-bc5b-2d5b8460bef1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0891960 May 25 16:08:25.927599 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74e2c317-925a-40a6-9422-b3ab62b1828d req-57ed73eb-1384-4801-b85a-6c28e6d2fe4c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0407650 May 25 16:08:26.453005 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-b2791dd2-eb4a-4b7b-8f3b-b2863d0c2fd8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:26.453631 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-b2791dd2-eb4a-4b7b-8f3b-b2863d0c2fd8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:26.549657 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-b2791dd2-eb4a-4b7b-8f3b-b2863d0c2fd8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:26.550924 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-b2791dd2-eb4a-4b7b-8f3b-b2863d0c2fd8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.2013617 May 25 16:08:26.695021 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Deleting port 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:08:26.695371 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:26.848813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] For port 5df9097f-ef39-4cbc-b791-fce24729f2ed, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=5df9097f-ef39-4cbc-b791-fce24729f2ed,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:26.848813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:27.159942 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5c92881-d4ea-4d29-ab30-3139e57abc5c req-a44662e5-5169-45e4-8acc-c893a5f4274f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:27.162715 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5c92881-d4ea-4d29-ab30-3139e57abc5c req-a44662e5-5169-45e4-8acc-c893a5f4274f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 947 time: 0.0938375 May 25 16:08:27.169026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:27.169882 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:27.174237 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Floating IP 36261b1e-35fc-430b-8736-29cd641cfe62 disassociated. External IP: 172.27.2.163, port: 5df9097f-ef39-4cbc-b791-fce24729f2ed. May 25 16:08:27.174237 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Calling delete_port for 5df9097f-ef39-4cbc-b791-fce24729f2ed owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:08:27.226199 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5c92881-d4ea-4d29-ab30-3139e57abc5c req-6f6d9bf7-f095-46b0-87b6-2984489f54bf tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=32ab79bb-f281-465f-93f1-440cbf7dec8c&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0537467 May 25 16:08:27.381245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:27.388001 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5317bdbc-c324-4bb0-a83f-d4626a2e4516 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:27.535612 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'Port': ['ID=5df9097f-ef39-4cbc-b791-fce24729f2ed,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:27.539685 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:27.740646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:08:27.953315 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:08:27.963509 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:08:27.975597 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:08:27.976686 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:28.038799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] No DVR routers for this DVR port 5df9097f-ef39-4cbc-b791-fce24729f2ed on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:08:28.041357 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:28.045424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-deleted', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:28.162738 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:08:28.201005 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-6f4b609b-fc11-4d65-9582-1d1bfacb9727 req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "DELETE /v2.0/ports/5df9097f-ef39-4cbc-b791-fce24729f2ed HTTP/1.1" status: 204 len: 173 time: 1.6439085 May 25 16:08:28.241566 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf', 'name': 'network-vif-deleted', 'tag': '5df9097f-ef39-4cbc-b791-fce24729f2ed', 'status': 'completed', 'code': 200} May 25 16:08:28.401849 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ad7f2a8f-2f42-4a0f-8c07-90c084414939 req-09153fcd-9e88-42dd-b202-0026aaea698b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.0482149 May 25 16:08:28.519053 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:28.523147 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:08:28.531596 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:08:28.532338 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:28.742558 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:08:29.090750 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:29.096160 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a3489eef-81f0-45cd-9844-44ec9f422421 None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:08:29.266235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:08:29.275408 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:08:29.319189 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 5df9097f-ef39-4cbc-b791-fce24729f2ed {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:08:29.319541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 5df9097f-ef39-4cbc-b791-fce24729f2ed not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:08:29.449961 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9b322978-d86b-481f-93bc-2be08669a7d5 None None] New status for floating IP 36261b1e-35fc-430b-8736-29cd641cfe62: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:08:29.632665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7908b24c-2cac-40fe-8d95-e2c477d342c1 req-9271c257-a7f2-44c9-aa83-d0e5a109b354 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=9f2b319c-fc7f-44d0-81c7-bbb8f355e1bf HTTP/1.1" status: 200 len: 210 time: 0.0585039 May 25 16:08:30.876648 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47746) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:30.948032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:30.979818 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.183244 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.187800 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4fb957b2-56d8-4992-80a7-429879de3777 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:31.205537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=32ab79bb-f281-465f-93f1-440cbf7dec8c,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:31.208246 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:31.308777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.310156 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "DELETE /v2.0/security-groups/32ab79bb-f281-465f-93f1-440cbf7dec8c HTTP/1.1" status: 204 len: 168 time: 0.4324481 May 25 16:08:31.313007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47770) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:31.333656 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.api.v2.resource [None req-c99e133d-5043-4573-9807-9968a105aec4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] show failed (client error): The resource could not be found. May 25 16:08:31.334967 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-c99e133d-5043-4573-9807-9968a105aec4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups/32ab79bb-f281-465f-93f1-440cbf7dec8c HTTP/1.1" status: 404 len: 347 time: 0.0207667 May 25 16:08:31.342596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47774) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:31.590928 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Deleting port 71ac5ece-56ff-4707-a295-aac78d41f825 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:08:31.591340 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.721861 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.722197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:31.855261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Calling delete_port for 71ac5ece-56ff-4707-a295-aac78d41f825 owned by network:floatingip {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:08:31.896748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47794) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:32.026733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.042087 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:32.045736 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.046159 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Usage tracker for resource:security_group_rule and tenant:7685803b752e4112ad2ea271ee69e169 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:32.049730 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:32.051712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7bacd431-aeb0-4688-b71d-3f596e5df9f9 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:32.055726 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:32.094929 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'Port': ['ID=71ac5ece-56ff-4707-a295-aac78d41f825,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:32.140918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.141853 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.142221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.142585 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.142963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.143320 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.143673 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.144035 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.144524 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.144718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.145242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.145412 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:08:32.147526 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.175752 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:32.202346 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.217173 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.218294 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "DELETE /v2.0/floatingips/36261b1e-35fc-430b-8736-29cd641cfe62 HTTP/1.1" status: 204 len: 168 time: 0.8745813 May 25 16:08:32.263108 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.276051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-557d65fe-41b6-4f63-88f2-2f822136cd88 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:32.317025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47808) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:32.323516 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=7685803b752e4112ad2ea271ee69e169&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4250579 May 25 16:08:32.339677 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=0d5d4a0e-0099-4f92-b6d7-95e8ad526ccf,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:32.346441 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.070s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:32.388292 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.421202 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.607629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47816) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:32.612901 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group': {'name': 'tempest-securitygroup--618442389', 'description': 'tempest-description--1405216284'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:32.618838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:32.622486 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.628642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:32.631281 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54be16ad-2a9c-4f79-9d91-4bce1e2327ba admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:32.639669 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:32.654816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:32.654816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=0d5d4a0e-0099-4f92-b6d7-95e8ad526ccf,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:32.663428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.664709 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/0d5d4a0e-0099-4f92-b6d7-95e8ad526ccf HTTP/1.1" status: 204 len: 168 time: 0.3465497 May 25 16:08:32.665601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:32.743105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.770672 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:32.777164 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:32.781111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:32.787611 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:32.863154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.969686 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:32.975223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b126184e-4a33-4051-8203-ba6f92ad9f9f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:33.019790 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroup': ['ID=d903c7b0-de69-47ad-94e9-88fec7d7a3f5,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:33.022708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:33.120523 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.122175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1779 time: 0.5134022 May 25 16:08:33.125637 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47824) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:33.131206 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group_rule': {'security_group_id': 'd903c7b0-de69-47ad-94e9-88fec7d7a3f5', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:33.137428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:33.142654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:33.147548 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:33.153474 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:33.326061 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.399519 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.553679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.580630 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-41c5baa6-f68b-4f75-abdb-36ed64a4a1ec tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:33.584954 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47834) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:33.604669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=747cd7c4-53a7-433f-ba94-03117045c2fa,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:33.607422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:33.621847 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.638493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:33.643031 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Usage tracker for resource:security_group_rule and tenant:b0bbf373296e4470a430ba5a78dc784e is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:33.646527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:33.653735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:33.695349 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.699162 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.5699368 May 25 16:08:33.699618 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47840) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:33.706657 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'security_group_rule': {'security_group_id': 'd903c7b0-de69-47ad-94e9-88fec7d7a3f5', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:33.721042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:33.726925 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:security_group_rule and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:33.730825 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:33.737747 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:33.795108 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.861642 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.941502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.942430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:33.949223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-fd631925-7e40-4fb4-9208-57119adbf769 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:33.994980 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2a9c2cd4-1fa0-4801-a768-f14959ec361f,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:33.998157 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:34.001097 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=b0bbf373296e4470a430ba5a78dc784e&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4141021 May 25 16:08:34.004505 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47846) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:34.060204 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.069967 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:34.103529 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=748ff763-af1e-4a66-b6e4-eca9467a5cba,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:34.116821 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:34.120307 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.157641 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.211011 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.211965 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ac1b3e68-fd53-4d15-87bc-ae5e1a6b4b9e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.5106664 May 25 16:08:34.215258 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47850) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:34.220638 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:34.226727 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:34.231697 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Usage tracker for resource:floatingip and tenant:690d1dab67d340598496a2b626fceaf2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:34.236356 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:34.248842 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:34.341159 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.346412 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c198819c-e851-4aab-af43-20b13a5bf6c4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:34.358506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2a9c2cd4-1fa0-4801-a768-f14959ec361f,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:34.362014 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:34.364001 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.365119 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2a9c2cd4-1fa0-4801-a768-f14959ec361f HTTP/1.1" status: 204 len: 168 time: 0.3591764 May 25 16:08:34.394818 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.545224 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:34.783222 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Allocated IP 172.27.2.219 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/31116110-ec95-4de5-8fa7-8c5f5faf56da) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:08:34.950516 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.072558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.076280 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:35.148400 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:08:35.151130 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:08:35.155816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:08:35.189846 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=31116110-ec95-4de5-8fa7-8c5f5faf56da,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:35.192843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:35.252963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.402306 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.636560 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.643904 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6b6e2deb-933f-4d79-851a-d702aac7c17e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.729584 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:35.735263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:35.900095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=31116110-ec95-4de5-8fa7-8c5f5faf56da,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:35.900095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.124s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:35.900095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:36.063378 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:36.072623 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 693 time: 1.8487623 May 25 16:08:36.312493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e150b646-65e4-4bcd-8c6e-79fa744db4c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:36.380747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e150b646-65e4-4bcd-8c6e-79fa744db4c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:36.381941 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e150b646-65e4-4bcd-8c6e-79fa744db4c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-securitygroup--618442389&fields=id&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 267 time: 0.1173546 May 25 16:08:36.550701 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-85cb9ca5-4094-4b24-92a2-9ba84959e465 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:36.552310 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-85cb9ca5-4094-4b24-92a2-9ba84959e465 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 817 time: 0.1630898 May 25 16:08:36.571917 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-1edd49dd-fc97-4b2f-84d2-5e83c245ee08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/quotas/690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 375 time: 0.0138648 May 25 16:08:36.680165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-da444071-3f3f-4d28-8ca8-18e0c66561d6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:36.799920 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-da444071-3f3f-4d28-8ca8-18e0c66561d6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:36.801293 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-da444071-3f3f-4d28-8ca8-18e0c66561d6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&fields=id HTTP/1.1" status: 200 len: 348 time: 0.2230906 May 25 16:08:37.092588 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-05a06a55-35ad-4f4b-ad24-f4db17d43dff service neutron] 10.222.0.82 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2756493 May 25 16:08:37.268413 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-27bc0fa0-9fc5-460b-9b34-7df1c8672188 service neutron] 10.222.0.82 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1644018 May 25 16:08:37.354751 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47920) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:37.540528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:37.568990 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1998b529-1b80-466b-b7b2-64e9b3536f6a req-9a29734b-a5bf-4a28-8420-f128486650b8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0615387 May 25 16:08:37.569981 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:37.574339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Usage tracker for resource:security_group_rule and tenant:49e3f82fc5314d0dba1f7bb20212e392 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:37.577490 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:37.584944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:37.683427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:37.767020 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:37.776717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c040e9bd-ca16-438e-88c5-f957c2e33ba6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:37.829833 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=49e3f82fc5314d0dba1f7bb20212e392&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4739046 May 25 16:08:37.831101 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5abf307f-731b-471a-b64b-b607bddaef03,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:37.833577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47942) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:37.838501 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:37.914538 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:37.944889 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:38.108628 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:38.114409 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-84ffce95-e83e-429e-8372-a8683f84194c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:38.128371 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5abf307f-731b-471a-b64b-b607bddaef03,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:38.131729 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:38.134112 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:38.141527 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5abf307f-731b-471a-b64b-b607bddaef03 HTTP/1.1" status: 204 len: 168 time: 0.3024597 May 25 16:08:38.160537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:38.437974 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-61ba0bc6-f7fe-4e9e-aab7-4577efad3d4a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:38.704210 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5389b9f7-71b9-4a43-9b44-53dd5a37512d req-4619917b-910f-439f-8899-1b9fa39a0ed8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0459001 May 25 16:08:39.100953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.134483 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 47964) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:39.156070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.172792 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:39.182773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Usage tracker for resource:security_group_rule and tenant:c8c393b1601f4b47a6f95b37276041e0 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:39.186501 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:39.194453 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:39.291366 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.409399 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.417270 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3b39bd19-f4d9-4477-9b36-16b69c72ebe3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:39.460967 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=c8c393b1601f4b47a6f95b37276041e0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3253372 May 25 16:08:39.461899 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=df213566-a0a2-40f5-9dba-0c5fc7af26e9,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:39.465034 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47968) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:39.465682 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:39.629221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.656193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.799762 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.803846 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e7c33c2-303b-47d2-810d-c3fbc623cf4f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:39.815946 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=df213566-a0a2-40f5-9dba-0c5fc7af26e9,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:39.820337 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:39.824037 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:39.825061 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/df213566-a0a2-40f5-9dba-0c5fc7af26e9 HTTP/1.1" status: 204 len: 168 time: 0.3588281 May 25 16:08:39.875330 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-09eaf430-fa3f-45d7-9435-bd1109de8cc1 req-c4e00e79-d6e5-4b51-b9ee-71e7f1c77bfd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0483584 May 25 16:08:40.302788 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 47986) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:40.464531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:40.480389 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:40.484382 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Usage tracker for resource:security_group_rule and tenant:88a4bfa640774940a86b39247a0f7fa7 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:40.487542 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:40.491529 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:40.496098 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:40.614568 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:40.689926 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:40.698108 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e82b57be-b6a1-4582-bcd0-49502a375025 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:40.740399 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=b0ed2b3d-9250-4675-9714-ac79e7f4f093,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:40.744699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=88a4bfa640774940a86b39247a0f7fa7&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4407582 May 25 16:08:40.745662 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:40.748372 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48000) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:40.819695 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:40.862170 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:41.005857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:41.010285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ca9a7d3c-2ace-47bb-bbce-fa0f69c382ed admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:41.028738 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=b0ed2b3d-9250-4675-9714-ac79e7f4f093,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:41.044747 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:41.062070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:41.068345 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ecbd470-aae2-4212-82cb-d760f2633fe5 req-a927f7d2-f3eb-4fd8-aca6-d1f7f5ccc23a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0718400 May 25 16:08:41.068786 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/b0ed2b3d-9250-4675-9714-ac79e7f4f093 HTTP/1.1" status: 204 len: 168 time: 0.3151081 May 25 16:08:41.108308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:42.243144 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0c04025-8ea6-40f9-a4ae-3f090a461973 req-d9f04c2a-fdcb-44e5-800e-924b138dccc4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0477812 May 25 16:08:43.176278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-19689da1-f968-48c3-8124-cb31cfb870c3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:43.177866 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-19689da1-f968-48c3-8124-cb31cfb870c3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 817 time: 0.1622839 May 25 16:08:43.260576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e8ee1015-a27a-4486-b23e-70464500f251 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:43.282999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e8ee1015-a27a-4486-b23e-70464500f251 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:security_group {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:43.285036 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-e8ee1015-a27a-4486-b23e-70464500f251 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 379 time: 0.0929041 May 25 16:08:43.303689 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'port': {'device_id': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'admin_state_up': True, 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'security_groups': ['d903c7b0-de69-47ad-94e9-88fec7d7a3f5']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:43.430464 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-71e0d4cd-fdc6-4eed-b4b2-986ab50abedf req-577e1e2a-4c79-441e-be94-c5d47e2f15d8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0653486 May 25 16:08:43.490091 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:43.503412 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:43.605714 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Made reservation on behalf of 690d1dab67d340598496a2b626fceaf2 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:08:43.610007 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:43.796915 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Allocated IP 172.20.0.11 (dc1f952c-66ab-4757-a4c9-eef7df57dea8/21c0b655-f786-4f4e-807c-e0313e0d89ae/0a449f37-2b2a-4bae-a488-8dcfceef1808) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:08:44.104956 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.122124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Transition to ACTIVE for port object 0a449f37-2b2a-4bae-a488-8dcfceef1808 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:08:44.269104 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.275195 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:44.348583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:08:44.351773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:08:44.356679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:08:44.392288 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:44.398256 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:44.435209 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:44.435900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.536822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48084) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:44.560997 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 934 time: 1.2614546 May 25 16:08:44.582089 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Request body: {'port': {'device_id': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:44.679672 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-5a4973d6-7e98-4ab8-a848-b7992285725d None None] Provisioning for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:08:44.689937 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-5a4973d6-7e98-4ab8-a848-b7992285725d None None] Provisioning complete for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:08:44.691154 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-5a4973d6-7e98-4ab8-a848-b7992285725d None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.709339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.732968 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:44.739406 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Usage tracker for resource:security_group_rule and tenant:4750d8513c2f4728a4d168d94436e3b2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:44.743862 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:44.759582 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d7b8d63b-5129-44fd-b11f-974642e8ba41 req-03dc5c57-d485-486f-9f1f-2f358dcb493c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:44.761111 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d7b8d63b-5129-44fd-b11f-974642e8ba41 req-03dc5c57-d485-486f-9f1f-2f358dcb493c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 932 time: 0.1661189 May 25 16:08:44.761750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.762508 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:44.779710 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-5a4973d6-7e98-4ab8-a848-b7992285725d None None] Port 0a449f37-2b2a-4bae-a488-8dcfceef1808 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:08:44.836552 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d7b8d63b-5129-44fd-b11f-974642e8ba41 req-f56a8940-aab3-4dc0-ba2c-c4da14c2a2ca tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0638270 May 25 16:08:44.906967 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:44.993798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.001432 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2c8629ce-e388-46f3-ad05-2cec8dadfcc3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:45.044359 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=4750d8513c2f4728a4d168d94436e3b2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5048811 May 25 16:08:45.045616 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d378603b-138c-4ab3-a4fd-fd1d9d56e1d9,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:45.047458 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48096) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:45.050200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-82e851a6-355f-46fe-8cdb-0ebd5183a89f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:45.151279 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.228883 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.269998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:08:45.281940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.298192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Transition to ACTIVE for port object 0a449f37-2b2a-4bae-a488-8dcfceef1808 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:08:45.403434 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.407459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-018a970b-8739-4118-9000-eeb7e5921b62 tempest-DeleteServersTestJSON-280370384 tempest-DeleteServersTestJSON-280370384-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:45.462070 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.551660 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:45.555368 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.148s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:45.558327 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-35003664-4d54-48c7-b47f-cdc83509ad18 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.090s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:45.596865 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.597993 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/d378603b-138c-4ab3-a4fd-fd1d9d56e1d9 HTTP/1.1" status: 204 len: 168 time: 0.5493045 May 25 16:08:45.633673 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d378603b-138c-4ab3-a4fd-fd1d9d56e1d9,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:45.638225 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.080s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:45.682604 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Attempting to bind port 0a449f37-2b2a-4bae-a488-8dcfceef1808 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:08:45.683024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Attempting to bind port 0a449f37-2b2a-4bae-a488-8dcfceef1808 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:08:45.683354 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Attempting to bind port 0a449f37-2b2a-4bae-a488-8dcfceef1808 on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:08:45.688765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Port 0a449f37-2b2a-4bae-a488-8dcfceef1808 on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:08:45.689345 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Attempting to bind port 0a449f37-2b2a-4bae-a488-8dcfceef1808 on network dc1f952c-66ab-4757-a4c9-eef7df57dea8 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:08:45.696299 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 8, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:08:45.696812 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Checking segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:08:45.697345 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Bound using segment: {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:08:45.697822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Bound port: 0a449f37-2b2a-4bae-a488-8dcfceef1808, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'e08db650-c58e-47d7-8f08-2ac947c7f402', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 943, 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:08:45.698249 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.798802 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:08:45.801949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:08:45.868765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.878585 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-bdf376e1-79cf-4433-8624-f299e94df5c2 tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:45.933502 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:45.934006 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:45.935542 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] 10.222.0.42 "PUT /v2.0/ports/0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 1094 time: 1.3584654 May 25 16:08:45.936056 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:45.964644 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:45.968053 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:46.050136 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'status': 'completed', 'code': 200} May 25 16:08:46.104166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d87f9ac-9c1b-48ec-a2ab-ffa255d240a1 req-7d91569b-fb83-487f-bc4f-fa753235c1f1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:46.105751 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d87f9ac-9c1b-48ec-a2ab-ffa255d240a1 req-7d91569b-fb83-487f-bc4f-fa753235c1f1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1262748 May 25 16:08:46.172636 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d87f9ac-9c1b-48ec-a2ab-ffa255d240a1 req-ce3b696c-3ede-458d-a569-1e62d653190d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0603607 May 25 16:08:46.208153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-d7b0dd23-75f9-4bc1-a816-74e3d2e6b0b3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:46.209990 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-d7b0dd23-75f9-4bc1-a816-74e3d2e6b0b3 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1097 time: 0.1194472 May 25 16:08:46.278815 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48116) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:46.299718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.302740 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-c1856892-b81e-41fd-8a22-08dcb913d1c9 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 217 time: 0.0751488 May 25 16:08:46.314861 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:08:46.319366 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Usage tracker for resource:security_group_rule and tenant:20a6f7dccda7480482e61e9e92d8f002 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:08:46.322586 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:08:46.329586 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:08:46.373247 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-6dfa963e-3890-4950-ba09-dfee8d778906 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:46.374806 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-6dfa963e-3890-4950-ba09-dfee8d778906 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0634179 May 25 16:08:46.413728 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.454442 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-62dff36a-fea8-48e3-817e-46c2e6a35c0c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:46.455924 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-62dff36a-fea8-48e3-817e-46c2e6a35c0c service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0724134 May 25 16:08:46.467714 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.477574 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d6b69c06-9192-4911-bb64-be85bcf29530 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:46.523852 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=20a6f7dccda7480482e61e9e92d8f002&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2430539 May 25 16:08:46.524746 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a681c2e6-467e-49d3-a03a-f930ad8be41b,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:46.527343 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48120) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:46.530054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:46.593071 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.621019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.699181 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-54a85249-70b4-4458-ac92-b0f08c5b9e71 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2363656 May 25 16:08:46.752307 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.756595 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c89915e3-7604-4d51-b998-fb8100593e38 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:46.771838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a681c2e6-467e-49d3-a03a-f930ad8be41b,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:46.774148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:46.788444 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:46.789223 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/a681c2e6-467e-49d3-a03a-f930ad8be41b HTTP/1.1" status: 204 len: 168 time: 0.2609208 May 25 16:08:46.861712 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0ec1c79b-1850-4af6-b813-8f7a71fda9d8 req-efe75f46-340c-4969-8e2e-73ea39c1df56 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1552806 May 25 16:08:47.176812 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-74e7171c-418c-4b6d-8d22-6f48be923c23 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:47.181866 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-74e7171c-418c-4b6d-8d22-6f48be923c23 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1097 time: 0.0801010 May 25 16:08:47.374252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-cd89c4a8-b8e5-4dc8-a1ce-953707e43bcc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:47.376929 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-cd89c4a8-b8e5-4dc8-a1ce-953707e43bcc service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.1894696 May 25 16:08:47.414256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f849bcb-5e01-4d1a-94d5-d5c8fa76694d req-b4d4907b-e30a-4354-95c7-a9b686440dca tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:47.423877 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6f849bcb-5e01-4d1a-94d5-d5c8fa76694d req-b4d4907b-e30a-4354-95c7-a9b686440dca tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1248493 May 25 16:08:47.478988 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6f849bcb-5e01-4d1a-94d5-d5c8fa76694d req-fc611239-b00d-42b4-b386-5503c44fc3c3 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0529218 May 25 16:08:47.521382 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-a97d1941-a049-466b-a0a6-a89e168e1743 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 217 time: 0.1372159 May 25 16:08:47.589320 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-b131ee13-ba43-40e3-b53f-8ad9bc6b31f7 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:47.590823 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-b131ee13-ba43-40e3-b53f-8ad9bc6b31f7 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0623004 May 25 16:08:47.683365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-cd7c5938-032d-409b-a66b-b6bbea78462c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:47.685032 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-cd7c5938-032d-409b-a66b-b6bbea78462c service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0868285 May 25 16:08:47.831828 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-985515dc-7385-49c1-a4b7-07a06879daeb service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1382010 May 25 16:08:47.982562 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-4a0c6e93-1eed-430b-a952-acc830436ba9 req-fb5deacd-4b05-4904-992d-d7f13b39906b service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1431408 May 25 16:08:48.700365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7e510035-d456-456e-a0bd-8987d1049c5a req-ff909d4c-c8f5-4acf-83af-82753fb9c656 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:48.702233 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e510035-d456-456e-a0bd-8987d1049c5a req-ff909d4c-c8f5-4acf-83af-82753fb9c656 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.0873444 May 25 16:08:48.764327 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e510035-d456-456e-a0bd-8987d1049c5a req-9d4b7526-a3ec-4bec-8b3b-a37dd4add7bd tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0556242 May 25 16:08:49.320410 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:08:49.619121 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:49.645881 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:08:49.712541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:49.730742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:08:49.747084 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:49.906129 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:49.907876 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:49.971302 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eab1d1b9-4a6a-414e-be9e-1b0adf8dc114 req-149968d7-875a-4d41-b1dd-df4411c78dc9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:49.973532 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eab1d1b9-4a6a-414e-be9e-1b0adf8dc114 req-149968d7-875a-4d41-b1dd-df4411c78dc9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 945 time: 0.0925503 May 25 16:08:49.978456 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:49.981632 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.074s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:50.021358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eab1d1b9-4a6a-414e-be9e-1b0adf8dc114 req-232af6a6-4680-4199-9421-efdad925d6b7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0414574 May 25 16:08:50.246859 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-929b8a15-16fc-4a3d-ba94-712f2c3a7d10 req-90d200a5-3690-49ae-a6f4-59ee8882b95d tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:50.248424 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-929b8a15-16fc-4a3d-ba94-712f2c3a7d10 req-90d200a5-3690-49ae-a6f4-59ee8882b95d tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0816696 May 25 16:08:50.283906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['0a449f37-2b2a-4bae-a488-8dcfceef1808'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:08:50.298282 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-929b8a15-16fc-4a3d-ba94-712f2c3a7d10 req-b22896ee-5d6a-44c4-8fac-bf14b537c230 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0437713 May 25 16:08:51.232277 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29591d80-7a91-4152-8446-185b5531264c req-dea18139-cf07-4c47-9571-ce8212a960e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:51.233791 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29591d80-7a91-4152-8446-185b5531264c req-dea18139-cf07-4c47-9571-ce8212a960e6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 945 time: 0.0966420 May 25 16:08:51.282085 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29591d80-7a91-4152-8446-185b5531264c req-2b0f51be-fb54-4d63-b69b-912591f21bc1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0412159 May 25 16:08:51.331430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:08:51.501811 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d48fe959-eec2-4f01-8e14-5a0c5a3d1538 req-78b141e4-2c5b-4aec-ba41-201445bc839f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:51.503252 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d48fe959-eec2-4f01-8e14-5a0c5a3d1538 req-78b141e4-2c5b-4aec-ba41-201445bc839f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0848403 May 25 16:08:51.524871 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:08:51.530061 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:08:51.530450 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:51.555399 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d48fe959-eec2-4f01-8e14-5a0c5a3d1538 req-68fb3157-ae0d-46c4-b572-ccc07de16cc6 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0461104 May 25 16:08:51.669251 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:51.699751 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:51.709676 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:51.810489 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:51.813316 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:51.884551 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'code': 200} May 25 16:08:51.887009 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:51.890020 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:52.484723 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97c852b3-e5ac-4a21-b687-87670b8f81d2 req-5428bc3f-3a86-4599-9516-f25d1a017c58 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:52.486175 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97c852b3-e5ac-4a21-b687-87670b8f81d2 req-5428bc3f-3a86-4599-9516-f25d1a017c58 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0893581 May 25 16:08:52.533353 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97c852b3-e5ac-4a21-b687-87670b8f81d2 req-96f61bfc-bff1-4982-b808-1ec8d26183c5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0413342 May 25 16:08:52.769277 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3b84222a-3387-4268-95e7-3eac7f6c3885 req-c63ca71f-ec73-4540-b720-99eaf576d1e3 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:52.770791 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3b84222a-3387-4268-95e7-3eac7f6c3885 req-c63ca71f-ec73-4540-b720-99eaf576d1e3 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0990667 May 25 16:08:52.819414 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3b84222a-3387-4268-95e7-3eac7f6c3885 req-e5b3a7ff-a64d-4f0f-84a1-70d91e7ab320 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0427203 May 25 16:08:53.744313 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-08d97d13-d99a-46b0-9ccd-f4a8b8a6690a req-843b93a0-8375-406d-b049-756132bc9d5c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:53.746278 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-08d97d13-d99a-46b0-9ccd-f4a8b8a6690a req-843b93a0-8375-406d-b049-756132bc9d5c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0870578 May 25 16:08:53.808769 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-08d97d13-d99a-46b0-9ccd-f4a8b8a6690a req-81350e35-786e-4bdc-83d6-5a0148894e57 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0555642 May 25 16:08:54.120930 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9b6b4a35-9f31-4c57-98cb-26de1583b7cd req-ac3c1dde-4e59-4bd0-84c4-8533a970fa6b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:54.122394 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9b6b4a35-9f31-4c57-98cb-26de1583b7cd req-ac3c1dde-4e59-4bd0-84c4-8533a970fa6b tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.1905808 May 25 16:08:54.169071 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9b6b4a35-9f31-4c57-98cb-26de1583b7cd req-d4c32cc7-dee7-4f61-ab92-6705c1f7afa7 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0407035 May 25 16:08:54.855262 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-0780e853-6cb7-45c6-93ff-d4a1c17a2178 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:54.856050 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-0780e853-6cb7-45c6-93ff-d4a1c17a2178 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Synchronizing usage tracker for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:08:54.914604 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ba505f8f-c572-4084-88c8-7487f7df5476 req-ed953e57-d663-48db-80bd-0b9b5fd96b3f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:54.917287 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ba505f8f-c572-4084-88c8-7487f7df5476 req-ed953e57-d663-48db-80bd-0b9b5fd96b3f service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1099 time: 0.1030622 May 25 16:08:54.946241 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-0780e853-6cb7-45c6-93ff-d4a1c17a2178 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Unset dirty status for tenant:e8bb2885c2ca420b852158159d30e3b4 on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:08:54.947625 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-0780e853-6cb7-45c6-93ff-d4a1c17a2178 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.2014496 May 25 16:08:55.024399 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ba505f8f-c572-4084-88c8-7487f7df5476 req-7c13cc0c-05ec-4931-99b8-7f4a8e110c1a service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0997002 May 25 16:08:55.062338 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3d5b0fe2-810a-4e02-a783-436681951428 req-029f4687-3905-4b2f-a6db-92741f358391 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:55.063949 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3d5b0fe2-810a-4e02-a783-436681951428 req-029f4687-3905-4b2f-a6db-92741f358391 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1253154 May 25 16:08:55.113197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Deleting port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:08:55.113535 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:55.119732 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3d5b0fe2-810a-4e02-a783-436681951428 req-5afc1fff-5933-4fe0-8f3e-3137338a1913 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0482132 May 25 16:08:55.257960 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] For port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,segment=NetworkSegment(f1cda128-299e-4975-835f-ff55ee624d0a),segment_id=f1cda128-299e-4975-835f-ff55ee624d0a)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:08:55.258317 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:55.370691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f1fefbb0-373f-4803-be1c-cc1b44322a60 req-f9f0633b-9d56-4e1a-a2a5-d5b4cbe0752a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:55.372243 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f1fefbb0-373f-4803-be1c-cc1b44322a60 req-f9f0633b-9d56-4e1a-a2a5-d5b4cbe0752a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 945 time: 0.0896723 May 25 16:08:55.391253 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Calling delete_port for 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:08:55.423507 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f1fefbb0-373f-4803-be1c-cc1b44322a60 req-4433fb9e-29d0-4df6-86ca-9dd62a8cd595 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/security-groups?id=bb0c097f-e86f-43cf-a661-bffb61d1fb33&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0450785 May 25 16:08:55.627919 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:55.633771 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:08:55.665974 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:08:55.784933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Pushing event deleted for resources: {'Port': ['ID=2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:08:55.788238 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:08:55.863973 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] No DVR routers for this DVR port 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:08:55.866662 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:55.869513 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'name': 'network-vif-deleted', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:55.940167 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '86d47b15-cbfa-4e41-92a7-9ce9415e7145', 'name': 'network-vif-deleted', 'tag': '2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44', 'status': 'completed', 'code': 200} May 25 16:08:55.982977 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-b8319f0f-4b09-4b9f-8c7d-797124cdc26b req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.42 "DELETE /v2.0/ports/2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 HTTP/1.1" status: 204 len: 173 time: 1.0298264 May 25 16:08:56.350137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6377120f-9b21-4778-bb55-58cab0a6ce51 req-143aa03a-f6d3-4a90-b7f6-e213055dd5c8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:56.351833 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6377120f-9b21-4778-bb55-58cab0a6ce51 req-143aa03a-f6d3-4a90-b7f6-e213055dd5c8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0977955 May 25 16:08:56.419956 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6377120f-9b21-4778-bb55-58cab0a6ce51 req-fc225305-cd16-4d49-8c5c-804b6790a77d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0598173 May 25 16:08:56.602336 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f42202ba-7480-4e13-a9fd-0e08f575523a req-3ab60035-38c0-421c-9252-ac0c8fc4e8fa tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0465293 May 25 16:08:57.617263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4033f2df-3c1e-4448-91c3-bfeffa3a5e44 req-13e77f2b-161f-4c69-9b0f-6fbc58f9bf06 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:57.618695 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4033f2df-3c1e-4448-91c3-bfeffa3a5e44 req-13e77f2b-161f-4c69-9b0f-6fbc58f9bf06 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0816731 May 25 16:08:57.669981 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4033f2df-3c1e-4448-91c3-bfeffa3a5e44 req-52269a94-e084-48a0-8ef0-0c199bc6b334 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0446346 May 25 16:08:57.770896 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9f28776a-7182-475c-bdc3-cf848f87beba req-4eeaa100-8750-4b47-a227-eeb90ee85990 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0434465 May 25 16:08:57.861068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0497822c-0aa4-4a78-b0fa-c8dc671fe7c5 req-13030107-b569-415d-a302-791a167bc564 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:57.862572 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0497822c-0aa4-4a78-b0fa-c8dc671fe7c5 req-13030107-b569-415d-a302-791a167bc564 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0839717 May 25 16:08:57.871242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48228) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:08:57.876573 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Request body: {'floatingip': {'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:08:57.955597 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:58.585628 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:58.663461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:58.968506 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aac7383a-7706-4cbb-9d45-4b35733ab5dc req-c9f88f6f-4caf-4e7f-a468-fe80376c2df2 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0526359 May 25 16:08:58.985114 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Floating IP 772310b5-e9cf-45eb-b710-39e7b132967e associated. External IP: 172.27.2.219, port: 0a449f37-2b2a-4bae-a488-8dcfceef1808. May 25 16:08:59.354570 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:08:59.443804 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:08:59.484653 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:08:59.485134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 2d95dae0-5a62-4d4e-b0cd-40a6c3c5bf44 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:08:59.569961 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:08:59.574816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:08:59.584932 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:08:59.602177 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:08:59.603329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:08:59.650581 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c3915a5d-632b-429a-b983-d1b8a4793b08 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "PUT /v2.0/floatingips/772310b5-e9cf-45eb-b710-39e7b132967e HTTP/1.1" status: 200 len: 992 time: 1.7738311 May 25 16:08:59.650581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:08:59.698935 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'status': 'completed', 'code': 200} May 25 16:08:59.837365 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:08:59.861267 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-035c808e-0058-4893-9540-a0f499b18bb9 req-928beffc-65bb-4e0e-ab2d-669c29a662e8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:08:59.865125 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-928beffc-65bb-4e0e-ab2d-669c29a662e8 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1099 time: 0.0793316 May 25 16:09:00.095198 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-035c808e-0058-4893-9540-a0f499b18bb9 req-e1e52b42-b986-4ab7-b7dc-91e3be23d14e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:00.097144 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-e1e52b42-b986-4ab7-b7dc-91e3be23d14e service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.2277043 May 25 16:09:00.211285 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0963d684-fd17-4c6e-b066-b77bae1b8d26 req-a5a1c933-99ad-4a3c-bb7d-3c80bf844c2f tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] 10.222.0.82 "GET /v2.0/ports?device_id=86d47b15-cbfa-4e41-92a7-9ce9415e7145 HTTP/1.1" status: 200 len: 210 time: 0.0915601 May 25 16:09:00.288938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '772310b5-e9cf-45eb-b710-39e7b132967e', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.219', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 465, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:35Z', 'updated_at': '2021-05-25T16:08:58Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:00.293315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:00.303094 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:00.303662 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:00.330117 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-f2f43482-483c-4f05-b7ea-a675d944b3f8 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 1000 time: 0.2220550 May 25 16:09:00.412166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-035c808e-0058-4893-9540-a0f499b18bb9 req-fd14c2ac-726f-4582-8936-8da0169d0137 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:00.414909 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-fd14c2ac-726f-4582-8936-8da0169d0137 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0775490 May 25 16:09:00.516361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-035c808e-0058-4893-9540-a0f499b18bb9 req-ba9e3480-896c-4dea-9027-f5fb1ffc2782 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:00.518354 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-ba9e3480-896c-4dea-9027-f5fb1ffc2782 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0968478 May 25 16:09:00.572672 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:00.685944 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-7c789595-57f3-4a71-8c27-c18f01ed30ea service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1608431 May 25 16:09:00.954376 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-035c808e-0058-4893-9540-a0f499b18bb9 req-1d4a4dea-5481-4030-adee-1f9b9c21e5ed service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2607892 May 25 16:09:01.204766 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '772310b5-e9cf-45eb-b710-39e7b132967e', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.219', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 465, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:35Z', 'updated_at': '2021-05-25T16:08:58Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:01.209441 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-5464b50a-8305-42ce-b587-c499efd301ac None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '772310b5-e9cf-45eb-b710-39e7b132967e', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'floating_ip_address': '172.27.2.219', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'fixed_ip_address': '172.20.0.11', 'status': 'DOWN', 'standard_attr_id': 465, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:08:35Z', 'updated_at': '2021-05-25T16:08:58Z', 'revision_number': 1, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:01.382761 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ec267bc-d771-49dc-92cf-d11bdecabe12 req-c15707d4-31d0-40e4-b512-20ab897d82ba tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:01.384208 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ec267bc-d771-49dc-92cf-d11bdecabe12 req-c15707d4-31d0-40e4-b512-20ab897d82ba tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1178391 May 25 16:09:01.453626 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ec267bc-d771-49dc-92cf-d11bdecabe12 req-db99286e-d399-449a-aa75-6ab30b9a1955 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0574443 May 25 16:09:01.473188 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4cd92a26-95e1-4f1a-96c0-43de646c5de9 None None] New status for floating IP 772310b5-e9cf-45eb-b710-39e7b132967e: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:09:02.662416 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-172591a6-6fd0-43d8-9ff0-3b73da08b798 req-a7c9a162-63b0-47b9-9c61-b955f2a4e810 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:02.663841 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-172591a6-6fd0-43d8-9ff0-3b73da08b798 req-a7c9a162-63b0-47b9-9c61-b955f2a4e810 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0852933 May 25 16:09:02.713788 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-172591a6-6fd0-43d8-9ff0-3b73da08b798 req-cb25ad91-2f6a-44b9-bf8c-203f74e2a455 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0438519 May 25 16:09:03.915393 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-39a7ed88-0499-4528-976b-8a360be10401 req-aa148e34-daa2-4445-8c07-56e81a87a7cc tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:03.916955 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-39a7ed88-0499-4528-976b-8a360be10401 req-aa148e34-daa2-4445-8c07-56e81a87a7cc tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0906105 May 25 16:09:03.970503 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-39a7ed88-0499-4528-976b-8a360be10401 req-eea21250-db00-4fac-af99-b106075faafd tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0429289 May 25 16:09:04.925827 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-4171c29b-af16-4c89-9278-2aaca7674b26 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:04.926545 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-4171c29b-af16-4c89-9278-2aaca7674b26 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99544) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:09:05.016649 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-4171c29b-af16-4c89-9278-2aaca7674b26 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99544) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:09:05.018019 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-4171c29b-af16-4c89-9278-2aaca7674b26 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.1835189 May 25 16:09:05.192093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Deleting port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:09:05.192093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:05.192432 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84ffd1c0-bc90-477e-9ef6-0c483946fb9f req-7e4dd704-a7da-4720-a342-ccfee74ca4ad tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:05.192432 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-84ffd1c0-bc90-477e-9ef6-0c483946fb9f req-7e4dd704-a7da-4720-a342-ccfee74ca4ad tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 945 time: 0.0964932 May 25 16:09:05.280699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-84ffd1c0-bc90-477e-9ef6-0c483946fb9f req-2b5c2f66-311f-479a-8998-48f73a2ffe42 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=e4aeeac6-6179-49e9-a625-03f354d51988&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0840199 May 25 16:09:05.380398 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] For port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:05.388523 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:05.658917 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-58ae2b63-8b40-453f-b3c2-a29d0598da2c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:05.786158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:05.786783 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:05.787330 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Floating IP 91c9ca1e-ed6d-4008-806e-89380d2d5382 disassociated. External IP: 172.27.2.179, port: 99600d03-17d9-4c12-98bd-6b71a8bbc8e3. May 25 16:09:05.787901 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Calling delete_port for 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 owned by compute:nova {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:09:06.186351 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:06.199112 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:06.244314 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'Port': ['ID=99600d03-17d9-4c12-98bd-6b71a8bbc8e3,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:06.247252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:06.268605 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:09:06.455335 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:09:06.456805 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b3c7593d-7374-4514-9851-e7217a6a83da req-56474d6f-c68c-4379-93a0-1d1976ecad32 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0577538 May 25 16:09:06.677448 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:06.688444 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:09:06.701336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:06.702718 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:06.768484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] No DVR routers for this DVR port 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 on host hv-cinder-78638 {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:09:06.771381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:06.774139 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'name': 'network-vif-deleted', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:09:06.835531 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '50dbcc6f-5059-42ad-b455-f2e09c9190dd', 'name': 'network-vif-deleted', 'tag': '99600d03-17d9-4c12-98bd-6b71a8bbc8e3', 'status': 'completed', 'code': 200} May 25 16:09:06.945356 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:06.988091 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-9d51c7cb-9d29-4e82-b98c-1b49676dc8e2 req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "DELETE /v2.0/ports/99600d03-17d9-4c12-98bd-6b71a8bbc8e3 HTTP/1.1" status: 204 len: 173 time: 1.9639764 May 25 16:09:07.327408 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:07.331718 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:07.341382 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:07.342012 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:08:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:07.537197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:07.670748 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aff669fb-f743-4be4-aa53-6d050a03454e req-9b0f7e2c-e5f2-4571-a351-399caaf28d2b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0842941 May 25 16:09:07.944478 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:07.948701 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8852139c-3509-477a-93d8-d58fc4c35b0d None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:08.141995 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:08.217329 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-bc2ecf7e-3e83-4103-afa8-6c0d3e74c433 None None] New status for floating IP 91c9ca1e-ed6d-4008-806e-89380d2d5382: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:09:08.455197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-76aea8bb-25c4-45a4-a4b7-245f1ff5ea16 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:08.913115 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2a6bf92f-77ad-43d3-8b74-23ed88c5cb2c req-149cf8c7-3511-42bd-8136-0b6743bdcd56 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.0451417 May 25 16:09:09.170111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:10.199263 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b39021f9-47fc-42f6-a9fd-fe51a2c920c0 req-d7746a7f-9782-4007-9f39-91b51a66b52d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=50dbcc6f-5059-42ad-b455-f2e09c9190dd HTTP/1.1" status: 200 len: 210 time: 0.1569896 May 25 16:09:10.506923 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.009712 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-bd2f6388-2070-4426-af45-a510dab6bdeb service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0791175 May 25 16:09:11.095843 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.127552 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-7dd9b5f6-6c7e-458f-ae28-43ecdfcb0c01 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:11.129319 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-7dd9b5f6-6c7e-458f-ae28-43ecdfcb0c01 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.0764358 May 25 16:09:11.317106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-8b25ba24-63fb-4f32-94b4-04c01cad10ac service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:11.318683 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-8b25ba24-63fb-4f32-94b4-04c01cad10ac service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.1636407 May 25 16:09:11.439802 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-41256987-1a02-42ff-8423-24ba83c1ce38 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 1000 time: 0.1135130 May 25 16:09:11.468606 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:09:11.503137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48396) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:11.508818 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:09:11.509405 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 99600d03-17d9-4c12-98bd-6b71a8bbc8e3 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:09:11.515278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-073bbff8-3237-4262-a3e8-0a6a4e78e82a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:11.518451 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-073bbff8-3237-4262-a3e8-0a6a4e78e82a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.0721092 May 25 16:09:11.593324 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.620131 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-952a14da-03c3-4d6e-8595-7cafea4d69b2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:11.622155 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-952a14da-03c3-4d6e-8595-7cafea4d69b2 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0967362 May 25 16:09:11.628444 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.781735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.784468 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-a31abe9b-55d2-49f0-99c7-30e442f55cd7 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1542714 May 25 16:09:11.786073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a3665646-f85b-488f-a54b-5ef54479b3e0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:11.802058 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=e4aeeac6-6179-49e9-a625-03f354d51988,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:11.804923 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:11.886504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:11.888380 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "DELETE /v2.0/security-groups/e4aeeac6-6179-49e9-a625-03f354d51988 HTTP/1.1" status: 204 len: 168 time: 0.3834395 May 25 16:09:11.891938 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48402) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:11.911657 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.api.v2.resource [None req-a6a3759d-8841-4caf-b7e1-92bb11069a28 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] show failed (client error): The resource could not be found. May 25 16:09:11.915732 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a6a3759d-8841-4caf-b7e1-92bb11069a28 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups/e4aeeac6-6179-49e9-a625-03f354d51988 HTTP/1.1" status: 404 len: 347 time: 0.0198426 May 25 16:09:11.915983 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48406) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:11.972388 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-cde141c7-e60b-4896-922d-eacb43f28dc7 req-7969d676-a861-4b34-b1f3-dcb4806439e5 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1790190 May 25 16:09:12.162220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Deleting port 3a0d7127-7f88-407f-9fd4-348a20a45471 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:09:12.162567 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:12.308476 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:12.308734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:12.529996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Calling delete_port for 3a0d7127-7f88-407f-9fd4-348a20a45471 owned by network:floatingip {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:09:12.833087 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:12.838993 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:12.893937 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'Port': ['ID=3a0d7127-7f88-407f-9fd4-348a20a45471,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:12.898659 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:13.012524 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:13.039670 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:13.039670 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "DELETE /v2.0/floatingips/91c9ca1e-ed6d-4008-806e-89380d2d5382 HTTP/1.1" status: 204 len: 168 time: 1.1181123 May 25 16:09:13.542114 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48432) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:13.548716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group': {'name': 'tempest-securitygroup--386095640', 'description': 'tempest-description--262722404'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:13.556031 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:13.561613 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:13.566393 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:13.574237 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:13.706159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:13.728125 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:13.736549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:13.739773 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:13.745798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:13.819488 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:13.902202 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:13.918784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-17977a94-dbf0-4203-b031-52df92228b37 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:13.960041 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroup': ['ID=3594719b-1bbe-47d2-8830-e02def651c58,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:13.963614 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:13.992045 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48442) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:14.057185 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.059655 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1778 time: 0.5162940 May 25 16:09:14.066737 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48450) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:14.078746 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group_rule': {'security_group_id': '3594719b-1bbe-47d2-8830-e02def651c58', 'protocol': 'tcp', 'ethertype': 'IPv4', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:14.094297 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:14.103149 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:14.111423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:14.127849 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:14.179095 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.196519 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:14.201101 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Usage tracker for resource:security_group_rule and tenant:853322e63db24c7c8a22c29c96c76ac8 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:14.204955 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:14.211964 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:14.247118 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.461507 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.478023 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.548369 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.551900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-de99cada-10ae-450b-a637-58e286b8dbb7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:14.588596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=0702e82f-2d57-415b-8a02-8460f8892332,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:14.591489 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:14.627824 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.638262 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-11161b4b-9e99-4bfa-bb2b-d6b6bd863ea1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:14.681763 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=853322e63db24c7c8a22c29c96c76ac8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.6881993 May 25 16:09:14.683647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=a5b49002-07da-4397-beba-b6e0db90e17f,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:14.684471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48454) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:14.687154 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:14.695727 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.697080 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-44e3cb2a-b614-4570-88b5-a964935c0160 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.6281381 May 25 16:09:14.699813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48458) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:14.706454 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'security_group_rule': {'security_group_id': '3594719b-1bbe-47d2-8830-e02def651c58', 'protocol': 'icmp', 'ethertype': 'IPv4', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:14.712525 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:14.719312 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:security_group_rule and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:14.723673 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:14.730823 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:14.781076 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.814226 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.828265 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:14.911425 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.009814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.010593 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.013140 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:15.014010 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a0f3bb64-eddc-4e7d-b88f-e061963d31e7 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:15.029321 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=a5b49002-07da-4397-beba-b6e0db90e17f,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:15.038119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.024s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:15.038558 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=9d01e15d-0cd4-4114-b1ce-c3bade15af97,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:15.050798 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.051192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.051777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.052139 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.052491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.053004 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.053365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.053720 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.054073 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.054423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.054731 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.054860 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.055137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:15.057419 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/a5b49002-07da-4397-beba-b6e0db90e17f HTTP/1.1" status: 204 len: 168 time: 0.3709211 May 25 16:09:15.062652 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:15.158439 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.160384 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4590313 May 25 16:09:15.163072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48468) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:15.168608 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:15.175104 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:15.179462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Usage tracker for resource:floatingip and tenant:06dda51337ff4f03a0a16af8eaf39a13 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:15.182427 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:15.189288 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:15.347106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.497723 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.665253 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Allocated IP 172.27.2.63 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/a222275e-740c-49f0-a5ad-d7f95cbc0fd2) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:15.846589 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.970197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:15.974291 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-25f552b0-d975-43be-9f61-599f82da4bf4 tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:16.042656 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:16.046344 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:16.050474 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:16.087156 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=a222275e-740c-49f0-a5ad-d7f95cbc0fd2,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:16.097249 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:16.145297 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.264123 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.500571 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.581817 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.588436 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a74e8b90-c795-404d-8b47-69be5b65568f tempest-VolumesNegativeTest-569621201 tempest-VolumesNegativeTest-569621201-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:16.761536 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=a222275e-740c-49f0-a5ad-d7f95cbc0fd2,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:16.767210 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.179s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:16.775263 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.930177 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:16.931861 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 692 time: 1.7676597 May 25 16:09:17.238534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-d8100ebb-9c81-4d05-8dcf-77ea70a85ee9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:09:17.358530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-d8100ebb-9c81-4d05-8dcf-77ea70a85ee9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:09:17.360106 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-d8100ebb-9c81-4d05-8dcf-77ea70a85ee9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-securitygroup--386095640&fields=id&tenant_id=06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 267 time: 0.1901822 May 25 16:09:17.539050 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-c002ee07-99a6-4d2f-8f01-3748f070479c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:17.540638 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-c002ee07-99a6-4d2f-8f01-3748f070479c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 811 time: 0.1673021 May 25 16:09:17.557775 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-afca6902-c52c-4b1f-9725-9cf2d46e188b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/quotas/06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 375 time: 0.0101252 May 25 16:09:17.663527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-8832e4fc-c3cf-47ae-b290-1ce5032e10e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:09:17.764612 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-8832e4fc-c3cf-47ae-b290-1ce5032e10e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:09:17.766355 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-8832e4fc-c3cf-47ae-b290-1ce5032e10e4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&fields=id HTTP/1.1" status: 200 len: 348 time: 0.1989303 May 25 16:09:17.929468 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-3e087af2-7c8b-4ba1-96a5-e7881f2c713f service neutron] 10.222.0.82 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1440909 May 25 16:09:18.166610 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-d7425380-bab8-497a-b2b9-e6217a2cbf2f service neutron] 10.222.0.82 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2307639 May 25 16:09:18.465349 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f90c554a-b827-421d-9d77-f51f970c99e4 req-d7700c4a-4370-4805-b875-eaa209f0529c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 210 time: 0.0535488 May 25 16:09:19.633700 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1b90fd24-75a5-416c-aa7d-d544491ce90c req-65860659-074e-41eb-9b47-59c2f304a800 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 210 time: 0.0439184 May 25 16:09:20.790616 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9ffc3802-aac9-40db-9fb8-78edbe43c628 req-0de32650-8650-4cb7-b19d-8ef5c009c329 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 210 time: 0.0482235 May 25 16:09:21.429233 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-5783fcbd-b632-4c4a-8374-9712cb799994 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:21.431740 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-5783fcbd-b632-4c4a-8374-9712cb799994 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 811 time: 0.1913126 May 25 16:09:21.493234 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-7b3b5914-a634-4b2a-8f36-472a7e7fd41f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=06dda51337ff4f03a0a16af8eaf39a13 HTTP/1.1" status: 200 len: 379 time: 0.0462298 May 25 16:09:21.504448 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'port': {'device_id': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'admin_state_up': True, 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'security_groups': ['3594719b-1bbe-47d2-8830-e02def651c58']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:21.718100 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:21.736837 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:21.838356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Made reservation on behalf of 06dda51337ff4f03a0a16af8eaf39a13 for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:09:21.843547 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:22.006023 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-445b13a5-64bf-427f-8c04-63ba6926e0a1 req-55799333-c2d1-43fc-b8c1-5b01c846cc84 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 210 time: 0.0812349 May 25 16:09:22.044834 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Allocated IP 172.20.0.4 (0105f026-2520-418a-a172-829f4b156413/c52f7a79-f047-4ca9-9f61-86cb7e470fe5/21c620fe-0dda-41cd-902b-278471e31c3e) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:22.368292 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:22.385623 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Transition to ACTIVE for port object 21c620fe-0dda-41cd-902b-278471e31c3e will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:22.519898 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:22.529847 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fdd84c83-0a01-4446-a524-d2e72795aeb4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:22.646914 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Network 0105f026-2520-418a-a172-829f4b156413 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:22.652735 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:22.660450 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:22.729663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event updated for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:22.729663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.180s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:22.746796 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:22.747336 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:22.914085 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.4135976 May 25 16:09:22.936539 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Request body: {'port': {'device_id': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:22.963274 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-f910c521-22a3-401e-a17b-e21083aa8e01 None None] Provisioning for port 21c620fe-0dda-41cd-902b-278471e31c3e completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:22.969130 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-f910c521-22a3-401e-a17b-e21083aa8e01 None None] Provisioning complete for port 21c620fe-0dda-41cd-902b-278471e31c3e triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:22.978938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f910c521-22a3-401e-a17b-e21083aa8e01 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:23.037798 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-f910c521-22a3-401e-a17b-e21083aa8e01 None None] Port 21c620fe-0dda-41cd-902b-278471e31c3e cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:09:23.085308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:23.279307 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0c362bde-bce0-48f7-a5d8-ad4ca7ebc7b3 req-067e2384-e126-4e34-bf1b-2a1754cfcbf9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:23.290260 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0c362bde-bce0-48f7-a5d8-ad4ca7ebc7b3 req-067e2384-e126-4e34-bf1b-2a1754cfcbf9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 931 time: 0.1540358 May 25 16:09:23.362770 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0c362bde-bce0-48f7-a5d8-ad4ca7ebc7b3 req-054ad86d-0365-4736-89fd-de71809dcf3f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0715606 May 25 16:09:23.644568 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 21c620fe-0dda-41cd-902b-278471e31c3e {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:09:23.652991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:23.670260 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Transition to ACTIVE for port object 21c620fe-0dda-41cd-902b-278471e31c3e will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:23.769990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:23.779347 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-baf3642b-de8b-404d-a59c-13cad251d982 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:24.106303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Pushing event updated for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:24.155192 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.376s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:24.232546 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Attempting to bind port 21c620fe-0dda-41cd-902b-278471e31c3e on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:24.233035 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Attempting to bind port 21c620fe-0dda-41cd-902b-278471e31c3e by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:24.233504 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Attempting to bind port 21c620fe-0dda-41cd-902b-278471e31c3e on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:24.241673 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Port 21c620fe-0dda-41cd-902b-278471e31c3e on network 0105f026-2520-418a-a172-829f4b156413 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:09:24.241996 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Attempting to bind port 21c620fe-0dda-41cd-902b-278471e31c3e on network 0105f026-2520-418a-a172-829f4b156413 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:24.248922 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:24.249256 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Checking segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:24.249653 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Bound using segment: {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:24.250092 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Bound port: 21c620fe-0dda-41cd-902b-278471e31c3e, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '4e483d36-603d-4ea7-a03e-3b742a52a6dc', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 932, 'network_id': '0105f026-2520-418a-a172-829f4b156413'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:24.250445 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:24.359687 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:24.363674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=21c620fe-0dda-41cd-902b-278471e31c3e,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:24.502229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:24.509042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:24.581275 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:24.582301 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:24.584751 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] 10.222.0.42 "PUT /v2.0/ports/21c620fe-0dda-41cd-902b-278471e31c3e HTTP/1.1" status: 200 len: 1093 time: 1.6561699 May 25 16:09:24.585583 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:09:24.630144 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Pushing event updated for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:24.639322 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.130s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:24.680878 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e', 'status': 'completed', 'code': 200} May 25 16:09:24.728497 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d6c2b0ae-8a55-4d54-8045-5376af1066bd req-8e2db4a0-7c41-4cb6-ba23-8f8d6cf121f1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:24.731551 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d6c2b0ae-8a55-4d54-8045-5376af1066bd req-8e2db4a0-7c41-4cb6-ba23-8f8d6cf121f1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 943 time: 0.1747508 May 25 16:09:24.845917 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d6c2b0ae-8a55-4d54-8045-5376af1066bd req-8d8424f1-5adc-41cb-b3ad-e455f1853bf4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1015141 May 25 16:09:24.851857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-ab60d495-98a7-49d4-88fa-a7eb58366527 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:24.854528 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-ab60d495-98a7-49d4-88fa-a7eb58366527 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 1096 time: 0.1321533 May 25 16:09:25.145959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-276c3961-5e21-4082-b692-759afe019f97 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:25.148525 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-276c3961-5e21-4082-b692-759afe019f97 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&shared=False HTTP/1.1" status: 200 len: 912 time: 0.2617743 May 25 16:09:25.391559 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-65d33c8d-2400-4c66-8644-79e1c2af572f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:25.393375 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0fe7f957-2c92-448f-8d07-c9a304840053 req-65d33c8d-2400-4c66-8644-79e1c2af572f service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.2372446 May 25 16:09:25.599078 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-cb322954-53da-4548-9dd4-b1aa6bb88376 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:25.600479 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-cb322954-53da-4548-9dd4-b1aa6bb88376 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 1096 time: 0.1215236 May 25 16:09:25.682197 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-6d4a28ac-04b8-4b0d-ac7e-84c2a501516d service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=21c620fe-0dda-41cd-902b-278471e31c3e HTTP/1.1" status: 200 len: 217 time: 0.0673029 May 25 16:09:25.853406 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-ab8df648-60c1-4403-9c49-ccf23646c476 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:25.854998 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-ab8df648-60c1-4403-9c49-ccf23646c476 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.1659198 May 25 16:09:25.952980 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-334c3f99-d92e-468e-8f1d-296d7635885c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:25.954704 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-334c3f99-d92e-468e-8f1d-296d7635885c service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0933490 May 25 16:09:26.127748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-685bf9b4-dd72-4897-8af7-f79154d1aea0 req-5d500a7b-0c69-4918-98b8-969af7ee3e2f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:26.129334 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-685bf9b4-dd72-4897-8af7-f79154d1aea0 req-5d500a7b-0c69-4918-98b8-969af7ee3e2f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 943 time: 0.1307344 May 25 16:09:26.133030 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-0e98d500-a7e6-4e27-abc4-b0072da543ed service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1710274 May 25 16:09:26.191196 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-685bf9b4-dd72-4897-8af7-f79154d1aea0 req-99a22dbd-555a-4059-a39e-25eff386c299 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0538008 May 25 16:09:26.335793 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-c7c4a9ea-f34a-47f7-a36e-8f8d742281c3 req-5e270250-69b2-4c09-81fe-0fdaad25ea92 service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1948588 May 25 16:09:27.427259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70fcf63f-c758-4b99-acb9-96582881c8f1 req-eb04eb76-31c3-40dd-89f5-3d40453fe25e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:27.428753 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70fcf63f-c758-4b99-acb9-96582881c8f1 req-eb04eb76-31c3-40dd-89f5-3d40453fe25e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 943 time: 0.0940804 May 25 16:09:27.484942 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70fcf63f-c758-4b99-acb9-96582881c8f1 req-6ddb2930-36b0-44b0-9817-8a1a809c89ae tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0490966 May 25 16:09:28.721571 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c812538b-c2e8-4598-ad75-265abb7f135d req-5f2a7c92-9ecc-4168-8e35-030971827a75 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:28.723181 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c812538b-c2e8-4598-ad75-265abb7f135d req-5f2a7c92-9ecc-4168-8e35-030971827a75 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 943 time: 0.1110268 May 25 16:09:28.773101 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c812538b-c2e8-4598-ad75-265abb7f135d req-44fed95f-de1e-461e-9811-fa8afc3b21c3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0432742 May 25 16:09:29.508836 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 21c620fe-0dda-41cd-902b-278471e31c3e details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:09:29.680434 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=21c620fe-0dda-41cd-902b-278471e31c3e,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:29.714336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '21c620fe-0dda-41cd-902b-278471e31c3e', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'port_id': '21c620fe-0dda-41cd-902b-278471e31c3e', 'mac_address': 'fa:16:3e:3f:a0:8a', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 932, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:09:29.763167 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:29.784495 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 21c620fe-0dda-41cd-902b-278471e31c3e {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:09:29.802369 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=21c620fe-0dda-41cd-902b-278471e31c3e,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:29.946565 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:29.948384 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:29.997694 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-281ca9b6-b238-4a97-95c0-3b2840e89049 req-5ff21d70-cc6b-4e5e-bc7a-5620bfb010a9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:30.000131 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-281ca9b6-b238-4a97-95c0-3b2840e89049 req-5ff21d70-cc6b-4e5e-bc7a-5620bfb010a9 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 943 time: 0.1112416 May 25 16:09:30.025604 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:30.034936 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.040666 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:30.043107 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:30.071894 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-281ca9b6-b238-4a97-95c0-3b2840e89049 req-a7157815-71d8-402d-a241-fe94040d46ec tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0602326 May 25 16:09:30.272282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['21c620fe-0dda-41cd-902b-278471e31c3e'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:09:31.338735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a779aa3c-cd79-4beb-bf0f-497ade8e30a7 req-d29da4b7-f59c-4a1f-b476-f6882a90f5ba tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:31.340280 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a779aa3c-cd79-4beb-bf0f-497ade8e30a7 req-d29da4b7-f59c-4a1f-b476-f6882a90f5ba tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 944 time: 0.1162999 May 25 16:09:31.397383 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a779aa3c-cd79-4beb-bf0f-497ade8e30a7 req-ddb362cc-324a-4333-81ac-45117b4e1715 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0482302 May 25 16:09:31.518852 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 21c620fe-0dda-41cd-902b-278471e31c3e up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:31.671960 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 21c620fe-0dda-41cd-902b-278471e31c3e completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:31.677595 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 21c620fe-0dda-41cd-902b-278471e31c3e triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:31.678741 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:31.817499 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:31.849108 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:09:31.862286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=21c620fe-0dda-41cd-902b-278471e31c3e,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:31.899543 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e', 'code': 200} May 25 16:09:31.981519 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:31.984973 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:32.083711 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:32.087406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:32.405402 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ee3d55da-54c1-467a-b91c-7731e0702285 req-5d3e534e-81d5-458e-b9e4-795776a5ff73 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:32.407217 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ee3d55da-54c1-467a-b91c-7731e0702285 req-5d3e534e-81d5-458e-b9e4-795776a5ff73 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 1098 time: 0.0830872 May 25 16:09:32.476792 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ee3d55da-54c1-467a-b91c-7731e0702285 req-8ecb955b-84c2-41fb-971c-f8482f83c45b service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0617762 May 25 16:09:32.617848 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7901f1ef-f1e1-444b-9111-91d7de5a3e49 req-215a04ce-2e16-46b2-bf14-c1e6bd6c32b7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:32.619460 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7901f1ef-f1e1-444b-9111-91d7de5a3e49 req-215a04ce-2e16-46b2-bf14-c1e6bd6c32b7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.0966887 May 25 16:09:32.671370 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7901f1ef-f1e1-444b-9111-91d7de5a3e49 req-e273cadd-92e3-4802-b3cc-47aead03a9ab tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0456684 May 25 16:09:33.937780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6d7ea046-ec84-4e2d-bfcf-2621385e305f req-e77a3df8-57f0-4558-a884-844ef89ed4cf tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:33.939783 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6d7ea046-ec84-4e2d-bfcf-2621385e305f req-e77a3df8-57f0-4558-a884-844ef89ed4cf tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1226852 May 25 16:09:33.999026 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6d7ea046-ec84-4e2d-bfcf-2621385e305f req-6e55d863-6c52-4683-9bb1-5552ba580d23 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0509031 May 25 16:09:34.361972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48640) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:34.434441 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Deleting subnet 2521d219-c255-41fe-80ad-cbccc4e9c1fe {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:09:34.438638 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:34.505575 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:34.578958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:34.975194 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:34.980352 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54c35220-e0b3-4c29-b6c3-b1fc7f3f0be3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:35.021941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=2521d219-c255-41fe-80ad-cbccc4e9c1fe,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:35.024892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.035072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.035652 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.035985 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.036627 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.036944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.037251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.037718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.038111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.038554 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.039058 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.039679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.040223 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:35.045978 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/2521d219-c255-41fe-80ad-cbccc4e9c1fe HTTP/1.1" status: 204 len: 168 time: 0.6833606 May 25 16:09:35.048270 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.067s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:35.053333 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48646) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:35.208742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.270226 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d0fdad5-e2b6-4130-8d70-0a2251da8a46 req-14edd3c2-e8d5-48c8-9878-7613d01d4c42 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:35.273029 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d0fdad5-e2b6-4130-8d70-0a2251da8a46 req-14edd3c2-e8d5-48c8-9878-7613d01d4c42 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1069682 May 25 16:09:35.328387 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d0fdad5-e2b6-4130-8d70-0a2251da8a46 req-27a887a1-7c16-49c3-b269-635fb82a0c2d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475976 May 25 16:09:35.373142 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.386646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['610dbdeb-d88d-45f7-b872-9563bbbe7c3f']}} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:35.402428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'f1cda128-299e-4975-835f-ff55ee624d0a') {'for_net_delete': True} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:35.402759 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'f1cda128-299e-4975-835f-ff55ee624d0a') {} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:35.424931 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3244641'] for segment, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.454723 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.471647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Releasing vlan 947 on physical network physnet1 to pool {{(pid=99544) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:09:35.472051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-3070910'] for segment, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: Traceback (most recent call last): May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: timer() May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: cb(*args, **kw) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: return func(*args, **kwargs) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: self._notify() May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: self.callback(batched_events) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: event.method(event) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: return f(self, *a, **k) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._get(url).json() May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: return self.request(url, 'GET', **kwargs) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:09:35.486359 dv-cinder-786384-6-iscsi neutron-server[99544]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:09:35.554618 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97bc4c20-cf12-415d-b27f-562788b1c545 req-b73e4613-d9c2-4717-8e96-aa16e97b8cb8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:35.556252 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97bc4c20-cf12-415d-b27f-562788b1c545 req-b73e4613-d9c2-4717-8e96-aa16e97b8cb8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1047628 May 25 16:09:35.576802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48654) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:35.585107 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Request body: {'floatingip': {'port_id': '21c620fe-0dda-41cd-902b-278471e31c3e'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:35.644524 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-5a5d22fb-7940-41a0-9a85-2459c70bbf88 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.672942 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.678278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:35.694689 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.710688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Pushing event deleted for resources: {'Network': ['ID=610dbdeb-d88d-45f7-b872-9563bbbe7c3f,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:35.714492 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.036s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:35.777131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:35.816197 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-eab7355d-77c4-4105-ae40-8566d4320235 admin admin] 10.222.0.82 "DELETE /v2.0/networks/610dbdeb-d88d-45f7-b872-9563bbbe7c3f HTTP/1.1" status: 204 len: 168 time: 0.7658048 May 25 16:09:36.437945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:36.564195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:36.640979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48660) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:36.801114 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5302e878-acc4-48ad-8573-d36c9f84e460 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=e8bb2885c2ca420b852158159d30e3b4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.1549199 May 25 16:09:36.801114 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Floating IP 2c3aee2e-3b67-4d25-b646-8b9cdeff04b2 associated. External IP: 172.27.2.63, port: 21c620fe-0dda-41cd-902b-278471e31c3e. May 25 16:09:36.802481 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48666) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:36.896074 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:36.942289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:37.151671 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:37.157713 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-031733eb-f817-468a-b4f8-b8a5522b9e33 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:37.172023 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bb0c097f-e86f-43cf-a661-bffb61d1fb33,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:37.195512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:37.298257 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:09:37.305135 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:37.306300 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4693de93-9d0b-4748-b234-b3a6309b3037 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/bb0c097f-e86f-43cf-a661-bffb61d1fb33 HTTP/1.1" status: 204 len: 168 time: 0.5024803 May 25 16:09:37.649674 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:37.657041 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:37.660160 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:09:37.670161 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:37.671222 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:37.716539 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f8156fcd-3a80-4523-8b72-f7967f654109 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "PUT /v2.0/floatingips/2c3aee2e-3b67-4d25-b646-8b9cdeff04b2 HTTP/1.1" status: 200 len: 990 time: 2.1381211 May 25 16:09:37.717592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:09:37.910727 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:37.953447 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e', 'status': 'completed', 'code': 200} May 25 16:09:38.085221 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-ae46571f-5cda-4930-a86b-947eb53bbc58 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:38.087325 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-ae46571f-5cda-4930-a86b-947eb53bbc58 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 1098 time: 0.0877380 May 25 16:09:38.246973 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:38.286370 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-ea344bf3-400e-4ead-bc19-4eb78e70938e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:38.288817 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-ea344bf3-400e-4ead-bc19-4eb78e70938e service neutron] 10.222.0.42 "GET /v2.0/networks?id=0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 200 len: 912 time: 0.1789064 May 25 16:09:38.415538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '2c3aee2e-3b67-4d25-b646-8b9cdeff04b2', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.63', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '21c620fe-0dda-41cd-902b-278471e31c3e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 503, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:3f:a0:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:09:16Z', 'updated_at': '2021-05-25T16:09:36Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:38.425613 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:38.428475 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-b7d1f241-ea1e-4afe-bbfd-6d8dcdcc138c service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=21c620fe-0dda-41cd-902b-278471e31c3e HTTP/1.1" status: 200 len: 998 time: 0.1314445 May 25 16:09:38.435685 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:38.436322 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:38.505521 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-76b04322-b9a7-4f18-afa9-db5755fa7a25 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:38.507029 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-76b04322-b9a7-4f18-afa9-db5755fa7a25 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 200 len: 850 time: 0.0678134 May 25 16:09:38.530582 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-6ca0a289-9447-4ab4-b366-971200fbe4d6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:38.608715 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-df42e768-6009-4c49-918f-398396d2764f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:38.610382 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-df42e768-6009-4c49-918f-398396d2764f service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=0105f026-2520-418a-a172-829f4b156413&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0966823 May 25 16:09:38.672246 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:38.679452 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48698) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:38.832689 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-830756281-network', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:38.837981 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:38.842890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Usage tracker for resource:network and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:38.846691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:38.854422 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:38.900299 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-8d39b87f-2110-4566-a9be-1885c75bb2ca service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2812734 May 25 16:09:38.936635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Made reservation on behalf of a1e31a264b7949bea6aa47cb7781106b for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:09:38.939556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:38.974632 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:38.997495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:39.013064 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:39.018116 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:39.029652 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:39.073823 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-55596853-1042-4b51-b4aa-14d5ff1bcbf6 req-d86408e3-464a-45e1-9fe1-ce49034846ed service neutron] 10.222.0.42 "GET /v2.0/networks/0105f026-2520-418a-a172-829f4b156413?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1672726 May 25 16:09:39.135522 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '2c3aee2e-3b67-4d25-b646-8b9cdeff04b2', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.63', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '21c620fe-0dda-41cd-902b-278471e31c3e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 503, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:3f:a0:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:09:16Z', 'updated_at': '2021-05-25T16:09:36Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:39.140570 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-79845dbc-07b4-4d40-9c92-046e01e9cfa3 None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '2c3aee2e-3b67-4d25-b646-8b9cdeff04b2', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'floating_ip_address': '172.27.2.63', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'port_id': '21c620fe-0dda-41cd-902b-278471e31c3e', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 503, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'mac_address': 'fa:16:3e:3f:a0:8a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:09:16Z', 'updated_at': '2021-05-25T16:09:36Z', 'revision_number': 1, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:39.168628 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.177018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.226410 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.232814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-54ac8bcd-528d-45e1-83d4-223baa92f93d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:39.259127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] vlan segment allocate from pool success with {'vlan_id': 902, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:09:39.276958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=41f65755-9a1f-492d-8410-648d24ee9269,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:39.281564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:39.282782 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.282954 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Added segment bd457928-332b-49b5-8448-df2758dfb4c0 of type vlan for network 6c61cff6-814b-4af0-8599-e026b705cc35 May 25 16:09:39.318577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.374397 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-881b56f2-144a-4022-b6be-357d314afbb5 None None] New status for floating IP 2c3aee2e-3b67-4d25-b646-8b9cdeff04b2: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:09:39.437504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.438858 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:39.439460 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:39.443110 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:39.548451 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Pushing event updated for resources: {'Network': ['ID=6c61cff6-814b-4af0-8599-e026b705cc35,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:39.551493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:39.560216 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 6c61cff6-814b-4af0-8599-e026b705cc35 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:09:39.622638 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 884 time: 0.9411910 May 25 16:09:39.627973 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48708) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:39.636076 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Request body: {'subnet': {'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-830756281-subnet', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:39.643062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:39.648606 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Usage tracker for resource:subnet and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:39.652886 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:39.660632 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:39.774443 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Made reservation on behalf of a1e31a264b7949bea6aa47cb7781106b for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:09:39.776869 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-3108046'] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.390701 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-3108443', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-3108056'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.394568 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:40.466367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.468703 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Pushing event updated for resources: {'Subnet': ['ID=a4d8980e-d38c-4165-af50-ed2f39dc708c,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:40.471812 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:40.480742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:40.483669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Network hostable DHCP agents. Network: 6c61cff6-814b-4af0-8599-e026b705cc35, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:09:40.597832 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:09:40.706181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 6c61cff6-814b-4af0-8599-e026b705cc35) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:40.715822 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-fc811eee-481c-4aec-b962-fe329e5a229f None None] Network 6c61cff6-814b-4af0-8599-e026b705cc35 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:09:40.717093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:40.717706 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.832767 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 847 time: 1.2028816 May 25 16:09:40.836186 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48720) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:40.842162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:40.847054 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:40.850889 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Usage tracker for resource:router and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:40.853948 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:40.861000 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:40.951788 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.952765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:40.972102 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.078674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-6c61cff6-814b-4af0-8599-e026b705cc35', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:09:41.080155 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.165595 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.271308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Allocated IP 172.20.0.2 (6c61cff6-814b-4af0-8599-e026b705cc35/a4d8980e-d38c-4165-af50-ed2f39dc708c/6d8cf16c-bdd9-414f-a6e4-12cff843fb64) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:41.350795 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.532679 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.548210 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Transition to ACTIVE for port object 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:41.569215 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Transition to ACTIVE for port object 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:41.718078 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.722550 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:41.794077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:41.794077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:41.794077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:41.803118 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Pushing event updated for resources: {'Port': ['ID=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:41.813456 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.830185 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.108s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:41.847426 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.899246 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Attempting to bind port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:41.899645 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Attempting to bind port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:41.900011 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Attempting to bind port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:41.912522 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:41.912989 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:41.913235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:41.913807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Bound using segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:41.914111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Bound port: 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:41.914485 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:42.016202 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] For port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:42.022031 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] For port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:42.090042 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:42.095629 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:42.123542 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Allocated IP 172.27.2.220 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/9c917315-3cdf-4b24-9114-5a5f1d99c173) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:42.266838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Pushing event updated for resources: {'Port': ['ID=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:42.280330 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.185s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:42.355501 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:42.464599 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:42.470625 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8c0045cd-fe2a-4e2e-905a-d19ec8e934cd tempest-AttachVolumeNegativeTest-884551932 tempest-AttachVolumeNegativeTest-884551932-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:42.538077 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:42.541131 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:42.552960 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:42.587942 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Pushing event updated for resources: {'Port': ['ID=9c917315-3cdf-4b24-9114-5a5f1d99c173,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:42.595178 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.126s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:43.123123 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:43.267973 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:09:43.301041 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-5d5cfbdf-2f30-4db7-8906-6f5909d5d48a None None] Network 6c61cff6-814b-4af0-8599-e026b705cc35 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:09:43.592302 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-bef20ac8-1cc9-47c3-9154-aa61c4681495 None None] Provisioning for port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:43.713051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:43.768719 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:43.991667 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:44.000284 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:44.002370 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.111555 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3dd3170c-fab9-4876-a47c-59e671d21c42 None None] Provisioning complete for port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:44.112113 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3dd3170c-fab9-4876-a47c-59e671d21c42 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.158224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99544) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:09:44.233907 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.374367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:44.384220 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3dd3170c-fab9-4876-a47c-59e671d21c42 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.518541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.524108 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:44.580552 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:44.605380 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:09:44.653754 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:44.653754 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:44.715091 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-82051c0e-736a-4fd5-8ab2-99005d08dad8 req-05d33e1a-676a-4fff-9cdc-eba2ec20084d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:44.717681 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-82051c0e-736a-4fd5-8ab2-99005d08dad8 req-05d33e1a-676a-4fff-9cdc-eba2ec20084d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1039617 May 25 16:09:44.723485 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:44.727682 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 923 time: 3.8883739 May 25 16:09:44.729478 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48754) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:44.745134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:44.750268 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-3dd3170c-fab9-4876-a47c-59e671d21c42 None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 16:09:44.750733 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-3dd3170c-fab9-4876-a47c-59e671d21c42 None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 16:09:44.753507 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.232s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:44.806941 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-82051c0e-736a-4fd5-8ab2-99005d08dad8 req-71fae5db-ca7a-4b97-9d09-ac3518a41765 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0813627 May 25 16:09:44.915200 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:45.238026 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:42Z', 'revision_number': 3, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:42Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:45.243051 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:45.243409 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Updating router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c port 9c917315-3cdf-4b24-9114-5a5f1d99c173 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:09:45.348671 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:45.597024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:45.711282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:45.877088 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Allocated IP 172.20.0.1 (6c61cff6-814b-4af0-8599-e026b705cc35/a4d8980e-d38c-4165-af50-ed2f39dc708c/921d6ca2-8279-4d74-8ee2-789091baf141) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:45.901760 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:45.902175 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:45.958154 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:45.983937 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Transition to ACTIVE for port object 9c917315-3cdf-4b24-9114-5a5f1d99c173 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:46.071379 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.072570 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.079501 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:46.100407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Transition to ACTIVE for port object 921d6ca2-8279-4d74-8ee2-789091baf141 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:46.115584 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-18c54a64-283f-495a-8c15-12dab0ad73b5 req-0b3be27d-74e4-49e5-b8f0-e77c2f71d569 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:46.117364 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-18c54a64-283f-495a-8c15-12dab0ad73b5 req-0b3be27d-74e4-49e5-b8f0-e77c2f71d569 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1160643 May 25 16:09:46.183042 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Pushing event updated for resources: {'Port': ['ID=9c917315-3cdf-4b24-9114-5a5f1d99c173,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:46.187211 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:46.187545 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Attempting to bind port 9c917315-3cdf-4b24-9114-5a5f1d99c173 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:46.187845 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Attempting to bind port 9c917315-3cdf-4b24-9114-5a5f1d99c173 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:46.188239 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Attempting to bind port 9c917315-3cdf-4b24-9114-5a5f1d99c173 on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:46.189210 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-18c54a64-283f-495a-8c15-12dab0ad73b5 req-8c3aab1d-763f-4d7d-b073-3d6f8d522e68 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0633051 May 25 16:09:46.194829 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:46.195300 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:46.196007 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.196117 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:46.196624 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Bound port: 9c917315-3cdf-4b24-9114-5a5f1d99c173, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:46.197063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.289175 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] For port 9c917315-3cdf-4b24-9114-5a5f1d99c173, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:46.294111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] For port 9c917315-3cdf-4b24-9114-5a5f1d99c173, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9c917315-3cdf-4b24-9114-5a5f1d99c173,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:46.443535 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.447917 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:46.478904 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.484339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:46.520820 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:46.524050 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:46.528306 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:46.564137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Pushing event updated for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:46.570011 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:46.570658 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:46.572894 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:46.608898 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:46.643182 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Pushing event updated for resources: {'Port': ['ID=9c917315-3cdf-4b24-9114-5a5f1d99c173,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:46.655476 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.172s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:46.819579 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:47.056446 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-60d1334f-fd1c-4a0b-a10c-1b7374129bd6 None None] Provisioning for port 921d6ca2-8279-4d74-8ee2-789091baf141 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:47.072574 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-60d1334f-fd1c-4a0b-a10c-1b7374129bd6 None None] Provisioning complete for port 921d6ca2-8279-4d74-8ee2-789091baf141 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:47.072948 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-60d1334f-fd1c-4a0b-a10c-1b7374129bd6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:47.141679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:47.175454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-60d1334f-fd1c-4a0b-a10c-1b7374129bd6 None None] Port 921d6ca2-8279-4d74-8ee2-789091baf141 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:09:47.202664 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:42Z', 'revision_number': 3, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:47.207232 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-a09f3505-e7ec-4963-ad04-ed9d0fb1e760 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:42Z', 'revision_number': 3, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:47.438325 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:47.519994 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:47.566708 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0d6eb486-198a-44e2-a6c3-0b6826bf543e req-1d0ecd9a-2ad5-4658-8f24-3379d66a6a4e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:47.568223 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d6eb486-198a-44e2-a6c3-0b6826bf543e req-1d0ecd9a-2ad5-4658-8f24-3379d66a6a4e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1888552 May 25 16:09:47.572805 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9e49ae31-e9a4-4e01-a5f7-8ff9d8a22456 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:47.680470 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:47.691953 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Pushing event updated for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:47.699998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.127s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:47.713209 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0d6eb486-198a-44e2-a6c3-0b6826bf543e req-2d178b89-8250-4819-a90e-c9f612338e83 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1384823 May 25 16:09:47.812406 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9c917315-3cdf-4b24-9114-5a5f1d99c173 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:47.880653 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:47.882087 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:48.003933 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 9c917315-3cdf-4b24-9114-5a5f1d99c173 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:48.012389 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9c917315-3cdf-4b24-9114-5a5f1d99c173 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:48.013664 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:48.201241 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:48.215099 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:09:48.230592 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 9c917315-3cdf-4b24-9114-5a5f1d99c173, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9c917315-3cdf-4b24-9114-5a5f1d99c173,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:48.233441 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-d4cfbf84-1055-484a-b7e9-ed972f3c79c2 None None] Provisioning complete for port 921d6ca2-8279-4d74-8ee2-789091baf141 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:48.233441 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d4cfbf84-1055-484a-b7e9-ed972f3c79c2 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:48.312990 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:48.324711 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:48.407706 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d4cfbf84-1055-484a-b7e9-ed972f3c79c2 None None] Port 921d6ca2-8279-4d74-8ee2-789091baf141 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:09:48.433315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=9c917315-3cdf-4b24-9114-5a5f1d99c173,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:48.437272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:48.483309 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:48.494162 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] 10.222.0.82 "PUT /v2.0/routers/f8e5a29f-4d37-4854-88b3-cfab3a35cd5c/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.7622705 May 25 16:09:48.994974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbc95113-930d-414f-8ae0-453c954111da req-115f06c1-3bb6-4c55-81f9-c2c34b647e2d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:48.997791 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbc95113-930d-414f-8ae0-453c954111da req-115f06c1-3bb6-4c55-81f9-c2c34b647e2d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1191900 May 25 16:09:49.055521 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbc95113-930d-414f-8ae0-453c954111da req-28d5154a-0dd0-4f0b-803e-7492393187c7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0511463 May 25 16:09:49.164204 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:09:49.174683 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:49.175352 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:49.397791 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:49.453509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48788) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:49.460438 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Request body: {'network': {'name': 'tempest-TestVolumeBootPattern-1294247851-network', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:49.468113 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:49.474385 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Usage tracker for resource:network and tenant:cdb3487ec5004d52a006a7780a5c92ea is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:49.478019 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:49.485328 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:49.556798 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Made reservation on behalf of cdb3487ec5004d52a006a7780a5c92ea for: {'network': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:09:49.559424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.577232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.605183 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:49.612572 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Usage tracker for resource:security_group_rule and tenant:cdb3487ec5004d52a006a7780a5c92ea is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:49.618699 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:49.626756 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:49.712180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9c917315-3cdf-4b24-9114-5a5f1d99c173 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:49.760139 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.808050 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 9c917315-3cdf-4b24-9114-5a5f1d99c173 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:49.808430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.832809 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.842491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a606be48-93d1-4fca-bed7-1d976e96360b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:49.854537 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:47Z', 'revision_number': 2, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:49.863490 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:49.864145 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Updating router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c port 921d6ca2-8279-4d74-8ee2-789091baf141 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:09:49.879012 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f168a92e-dc40-4e0c-973d-75bb5df77ee1,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:49.884152 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:49.896600 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] vlan segment allocate from pool success with {'vlan_id': 919, 'physical_network': 'physnet1'} {{(pid=99544) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:09:49.909395 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.909886 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.segments_db [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Added segment 1afdd637-4008-4148-a334-b60773d99ad1 of type vlan for network 2df16f06-2058-4b69-828c-2c380456e335 May 25 16:09:49.953826 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:49.956158 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.112453 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.115518 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:50.115518 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.121172 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:50.278348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Pushing event updated for resources: {'Network': ['ID=2df16f06-2058-4b69-828c-2c380456e335,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:50.282470 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.161s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:50.290172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-41bea5b1-6350-4366-9469-f14d6ea71e96 req-daeb1c4d-91c5-4275-a7e0-d9fbc3449d74 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:50.292379 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 2df16f06-2058-4b69-828c-2c380456e335 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:09:50.294197 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-41bea5b1-6350-4366-9469-f14d6ea71e96 req-daeb1c4d-91c5-4275-a7e0-d9fbc3449d74 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0922694 May 25 16:09:50.350069 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-41bea5b1-6350-4366-9469-f14d6ea71e96 req-4775305f-7566-4455-bc30-b6a458239166 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0496352 May 25 16:09:50.358817 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 885 time: 0.9040999 May 25 16:09:50.362360 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48796) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:50.374151 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Request body: {'subnet': {'network_id': '2df16f06-2058-4b69-828c-2c380456e335', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestVolumeBootPattern-1294247851-subnet', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'ip_version': 4}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:50.385638 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:50.390902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Usage tracker for resource:subnet and tenant:cdb3487ec5004d52a006a7780a5c92ea is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:50.394065 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:50.400164 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:50.431047 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.447892 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Transition to ACTIVE for port object 921d6ca2-8279-4d74-8ee2-789091baf141 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:50.477094 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Made reservation on behalf of cdb3487ec5004d52a006a7780a5c92ea for: {'subnet': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:09:50.479701 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-3108046'] for subnet, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.540753 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.545880 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:50.732934 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Pushing event updated for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:50.790994 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.791352 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.791793 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.792304 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.793552 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.793552 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.793895 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.795040 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.795646 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.796222 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.796779 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.797341 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:09:50.831803 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.286s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:50.870319 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Attempting to bind port 921d6ca2-8279-4d74-8ee2-789091baf141 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:50.870817 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Attempting to bind port 921d6ca2-8279-4d74-8ee2-789091baf141 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:50.871366 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Attempting to bind port 921d6ca2-8279-4d74-8ee2-789091baf141 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:50.878558 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:50.878967 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:50.879492 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.879895 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Bound using segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:50.880381 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Bound port: 921d6ca2-8279-4d74-8ee2-789091baf141, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:50.880761 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:50.979112 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] For port 921d6ca2-8279-4d74-8ee2-789091baf141, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:50.982289 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] For port 921d6ca2-8279-4d74-8ee2-789091baf141, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=921d6ca2-8279-4d74-8ee2-789091baf141,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:51.209075 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:51.213836 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-3108443', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-3108056'] for subnet, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:51.215468 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:51.223105 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ce39ea7f-cf7b-4b1f-8ddf-d947d93f3f09 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:51.310473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Pushing event updated for resources: {'Subnet': ['ID=1e6dd188-e084-4f58-b243-e638c41a8cc8,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:51.315676 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:51.321940 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:51.322425 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:51.330513 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Checking segment: {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:51.331014 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Network hostable DHCP agents. Network: 2df16f06-2058-4b69-828c-2c380456e335, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99544) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:09:51.465321 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Pushing event updated for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:51.470980 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5e68a5dc-3651-458a-934c-f64ecbf6335f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.255s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:51.556245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Network 2df16f06-2058-4b69-828c-2c380456e335 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99544) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:09:51.590005 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:51.596509 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db7214cd-17e2-43b8-8d34-3aed65ef864b req-00b1c36b-9b2a-4b47-8478-bf4ddc3c7421 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:51.598365 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db7214cd-17e2-43b8-8d34-3aed65ef864b req-00b1c36b-9b2a-4b47-8478-bf4ddc3c7421 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1099939 May 25 16:09:51.661017 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db7214cd-17e2-43b8-8d34-3aed65ef864b req-9617020b-41ce-467b-9483-257578424802 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0461793 May 25 16:09:51.678114 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 2df16f06-2058-4b69-828c-2c380456e335) {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:51.696742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-251fae2a-9750-4398-a35b-8faf429271a0 None None] Network 2df16f06-2058-4b69-828c-2c380456e335 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:09:51.700452 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:51.701092 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:51.855575 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-7d306726-1065-4b77-a941-4ad3ff24cd2f admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 848 time: 1.4913571 May 25 16:09:51.859420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48804) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:51.865268 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Request body: {'router': {'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:09:51.871197 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:09:51.876356 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Usage tracker for resource:router and tenant:cdb3487ec5004d52a006a7780a5c92ea is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:09:51.880681 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:09:51.887983 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:09:51.977778 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-2df16f06-2058-4b69-828c-2c380456e335', 'network_id': '2df16f06-2058-4b69-828c-2c380456e335', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'fixed_ips': [{'subnet_id': '1e6dd188-e084-4f58-b243-e638c41a8cc8'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:09:51.979142 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:51.989517 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:50Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:51.994010 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-655e4b19-0e7e-42ec-b9be-1668d048e0e6 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:50Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:52.005590 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.011986 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.030259 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.207583 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Allocated IP 172.20.0.2 (2df16f06-2058-4b69-828c-2c380456e335/1e6dd188-e084-4f58-b243-e638c41a8cc8/d0973574-d7d4-472e-a87f-448d82cb5ed3) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:52.493054 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.566966 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.584851 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Transition to ACTIVE for port object d0973574-d7d4-472e-a87f-448d82cb5ed3 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:52.612157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Transition to ACTIVE for port object d0973574-d7d4-472e-a87f-448d82cb5ed3 will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:52.717909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.726747 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:52.817622 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Network 2df16f06-2058-4b69-828c-2c380456e335 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:52.821340 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:52.830691 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:52.917532 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e463837b-207a-49f4-9a43-6f8435a6665e req-0a1b38d6-6fa2-4446-aeb4-09cf77a34434 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:52.920474 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e463837b-207a-49f4-9a43-6f8435a6665e req-0a1b38d6-6fa2-4446-aeb4-09cf77a34434 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1121075 May 25 16:09:52.979258 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e463837b-207a-49f4-9a43-6f8435a6665e req-87d06069-c795-4d33-a955-6237ccba98a2 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0527279 May 25 16:09:52.981109 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Pushing event updated for resources: {'Port': ['ID=d0973574-d7d4-472e-a87f-448d82cb5ed3,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:52.985435 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Attempting to bind port d0973574-d7d4-472e-a87f-448d82cb5ed3 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:52.985813 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Attempting to bind port d0973574-d7d4-472e-a87f-448d82cb5ed3 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:52.986181 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Attempting to bind port d0973574-d7d4-472e-a87f-448d82cb5ed3 on network 2df16f06-2058-4b69-828c-2c380456e335 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:52.988641 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.263s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:52.993157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:52.993540 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Checking segment: {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:52.993985 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:52.994444 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Bound using segment: {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:52.995005 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Bound port: d0973574-d7d4-472e-a87f-448d82cb5ed3, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:52.995418 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.123053 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.131370 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] For port d0973574-d7d4-472e-a87f-448d82cb5ed3, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:53.134938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] For port d0973574-d7d4-472e-a87f-448d82cb5ed3, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d0973574-d7d4-472e-a87f-448d82cb5ed3,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:53.168530 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.260590 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.270998 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:53.365491 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Pushing event updated for resources: {'Port': ['ID=d0973574-d7d4-472e-a87f-448d82cb5ed3,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:53.369100 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.098s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:53.382692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Allocated IP 172.27.2.130 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/00083874-f8bf-4dc1-b252-9739e23df60e) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:53.593341 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.717042 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:53.722066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-73eaa181-4727-4398-b83a-62d44fc50975 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:53.790911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:53.794532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:53.796818 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:53.812910 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d0973574-d7d4-472e-a87f-448d82cb5ed3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:53.831692 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Pushing event updated for resources: {'Port': ['ID=00083874-f8bf-4dc1-b252-9739e23df60e,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:53.839705 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:53.951533 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port d0973574-d7d4-472e-a87f-448d82cb5ed3 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:53.957564 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 921d6ca2-8279-4d74-8ee2-789091baf141 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:54.099706 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 921d6ca2-8279-4d74-8ee2-789091baf141 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:54.105731 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 921d6ca2-8279-4d74-8ee2-789091baf141 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:54.106119 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:54.259632 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:54.284809 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-daae5aa9-98a4-4fdf-83dc-49f2f2de3258 req-21cc5784-cce2-490a-a6bb-6f1dc901cfaa tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:54.298091 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-daae5aa9-98a4-4fdf-83dc-49f2f2de3258 req-21cc5784-cce2-490a-a6bb-6f1dc901cfaa tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1676986 May 25 16:09:54.317154 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 921d6ca2-8279-4d74-8ee2-789091baf141, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=921d6ca2-8279-4d74-8ee2-789091baf141,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:54.349472 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-daae5aa9-98a4-4fdf-83dc-49f2f2de3258 req-bcf44b2b-2969-453f-b4bb-f6871b4c8639 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0484238 May 25 16:09:54.433484 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-6b558794-84e5-4fe7-b221-17d0e5d0e270 None None] Network 2df16f06-2058-4b69-828c-2c380456e335 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:09:54.472087 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:54.564393 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:54.581659 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:54.771560 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Provisioning for port d0973574-d7d4-472e-a87f-448d82cb5ed3 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:54.784426 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Provisioning complete for port d0973574-d7d4-472e-a87f-448d82cb5ed3 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:54.784807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:54.851250 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:54.865266 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.284s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:55.001115 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:55.042252 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:55.087421 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] For port d0973574-d7d4-472e-a87f-448d82cb5ed3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d0973574-d7d4-472e-a87f-448d82cb5ed3,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:55.174150 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:55.177737 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:55.261942 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Pushing event updated for resources: {'Port': ['ID=d0973574-d7d4-472e-a87f-448d82cb5ed3,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:55.265698 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.088s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:55.294650 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Router dc208968-f643-4678-bf5f-5f1da544823d is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99544) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:09:55.296160 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-da1c93a2-8128-444b-8142-cf2ce1738d28 None None] Provisioning complete for port d0973574-d7d4-472e-a87f-448d82cb5ed3 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:55.296533 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-da1c93a2-8128-444b-8142-cf2ce1738d28 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:55.489681 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:55.500159 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Sync routers for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:09:55.511340 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:55.512483 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:55.573089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a2ab08f7-75f1-4b9c-891a-8a4b90ac28de req-3ec2d9fd-d553-4c60-a84b-2dddd0eee1c9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:55.574537 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a2ab08f7-75f1-4b9c-891a-8a4b90ac28de req-3ec2d9fd-d553-4c60-a84b-2dddd0eee1c9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0919721 May 25 16:09:55.623637 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a2ab08f7-75f1-4b9c-891a-8a4b90ac28de req-8980a572-9701-4bbd-b210-0f1c94cbe05a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0426052 May 25 16:09:55.648567 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:55.652997 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-7473d69c-a861-43a6-83db-51074da70a15 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 924 time: 3.7921133 May 25 16:09:55.654287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48820) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:09:55.697799 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:55.743942 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d0973574-d7d4-472e-a87f-448d82cb5ed3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:55.993137 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port d0973574-d7d4-472e-a87f-448d82cb5ed3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:55.994257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.176735 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 921d6ca2-8279-4d74-8ee2-789091baf141 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:56.210207 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 3, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:09:53Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:56.229824 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Checking router: dc208968-f643-4678-bf5f-5f1da544823d for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:09:56.230170 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Updating router dc208968-f643-4678-bf5f-5f1da544823d port 00083874-f8bf-4dc1-b252-9739e23df60e binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:09:56.298491 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.351192 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 921d6ca2-8279-4d74-8ee2-789091baf141 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:56.351573 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.501626 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.761798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Allocated IP 172.20.0.1 (2df16f06-2058-4b69-828c-2c380456e335/1e6dd188-e084-4f58-b243-e638c41a8cc8/6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:09:56.839439 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.852537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Transition to ACTIVE for port object 00083874-f8bf-4dc1-b252-9739e23df60e will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:56.927892 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:56.933634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:56.986470 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3df39831-4129-4aff-9a25-fc76dcfca674 req-1eb898a0-774c-458d-ad8e-7a61050bfdd6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:56.986984 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3df39831-4129-4aff-9a25-fc76dcfca674 req-1eb898a0-774c-458d-ad8e-7a61050bfdd6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.2315464 May 25 16:09:57.023074 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Attempting to bind port 00083874-f8bf-4dc1-b252-9739e23df60e on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:09:57.023656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.023783 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Attempting to bind port 00083874-f8bf-4dc1-b252-9739e23df60e by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:09:57.023900 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Attempting to bind port 00083874-f8bf-4dc1-b252-9739e23df60e on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:09:57.035153 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:09:57.035541 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:09:57.035955 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.036356 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:09:57.036838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Bound port: 00083874-f8bf-4dc1-b252-9739e23df60e, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:09:57.037219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.059529 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Transition to ACTIVE for port object 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:09:57.063022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Pushing event updated for resources: {'Port': ['ID=00083874-f8bf-4dc1-b252-9739e23df60e,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:57.070561 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:57.075582 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3df39831-4129-4aff-9a25-fc76dcfca674 req-892d9034-6da3-4e4a-ba85-a5e84b12aadb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0701981 May 25 16:09:57.148725 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] For port 00083874-f8bf-4dc1-b252-9739e23df60e, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:09:57.154853 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] For port 00083874-f8bf-4dc1-b252-9739e23df60e, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=00083874-f8bf-4dc1-b252-9739e23df60e,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:09:57.163421 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.167970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-89a159ae-7228-4f13-9588-97d99cc5e84d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:57.239776 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Network 2df16f06-2058-4b69-828c-2c380456e335 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:57.242350 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:57.246497 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:09:57.273538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.277919 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Pushing event updated for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:57.285588 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.117s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:57.293013 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-b21ce076-7ee9-4b80-8246-6ee350f5a680 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:57.312525 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.392889 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:09:57.393946 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:57.488389 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Pushing event updated for resources: {'Port': ['ID=00083874-f8bf-4dc1-b252-9739e23df60e,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:57.501973 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.209s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:57.640357 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:09:57.671825 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:09:57.777946 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.resource_extend [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] It took 0.12 seconds to run function 'neutron_lib.db.resource_extend.apply_funcs' {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py:386}} May 25 16:09:57.791136 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-639a59f8-3f3d-4633-ac05-63456fed5fd4 None None] Provisioning for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:09:57.794859 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:57.880779 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-639a59f8-3f3d-4633-ac05-63456fed5fd4 None None] Provisioning complete for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:57.881446 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-639a59f8-3f3d-4633-ac05-63456fed5fd4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.033106 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-639a59f8-3f3d-4633-ac05-63456fed5fd4 None None] Port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:09:58.087080 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:58.115778 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:09:58.280692 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.301809 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: ACTIVE current_port_status: BUILD port_id 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:09:58.325295 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:09:58.332679 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 3, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:09:58.338402 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-833d69c4-90ca-47b7-8848-a2410811fe4c None None] The sync data for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi is: [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 3, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:09:58.348079 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.380649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c12fe1e-1232-4c1b-9d03-d0193ef74867 req-a7639eb5-8485-4d8a-9b0e-33fb13f1a98b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:58.380649 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c12fe1e-1232-4c1b-9d03-d0193ef74867 req-a7639eb5-8485-4d8a-9b0e-33fb13f1a98b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1377709 May 25 16:09:58.397443 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.400683 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-7b56f525-03f8-4add-9862-ae0dfdee92a7 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:58.436097 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.476414 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:09:58.529303 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=6']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:58.537827 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.137s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:58.553824 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c12fe1e-1232-4c1b-9d03-d0193ef74867 req-f3e2f7c1-3e90-49d8-a4b1-b7fa21b3d1a5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1629159 May 25 16:09:58.566632 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['0a449f37-2b2a-4bae-a488-8dcfceef1808'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:09:58.577528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:58.585726 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Pushing event updated for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:09:58.591897 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.116s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:09:58.801827 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Network 2df16f06-2058-4b69-828c-2c380456e335 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:09:58.804293 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:09:59.075668 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-fbf7707e-5b92-49f7-a53c-64928c92c5d9 None None] Provisioning complete for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:59.077320 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-fbf7707e-5b92-49f7-a53c-64928c92c5d9 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:59.182576 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Router dc208968-f643-4678-bf5f-5f1da544823d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:09:59.200634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-fbf7707e-5b92-49f7-a53c-64928c92c5d9 None None] Port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:09:59.430170 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:09:59.441724 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] 10.222.0.82 "PUT /v2.0/routers/dc208968-f643-4678-bf5f-5f1da544823d/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.7862260 May 25 16:09:59.657062 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:59.800910 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:09:59.801620 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:09:59.853401 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 00083874-f8bf-4dc1-b252-9739e23df60e up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:09:59.863676 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-00ad740f-0c62-4617-8c5d-e4b92a6bf0a0 req-f7ae90e1-d1ef-44b9-92f5-98fee3a58701 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:09:59.871839 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-00ad740f-0c62-4617-8c5d-e4b92a6bf0a0 req-f7ae90e1-d1ef-44b9-92f5-98fee3a58701 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 945 time: 0.1280282 May 25 16:09:59.955927 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-00ad740f-0c62-4617-8c5d-e4b92a6bf0a0 req-d20a9f6c-5dbd-486f-8190-2284782879eb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0830193 May 25 16:10:00.090640 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 00083874-f8bf-4dc1-b252-9739e23df60e completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:10:00.101197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 00083874-f8bf-4dc1-b252-9739e23df60e triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:00.101840 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.130479 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.196134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:00.211748 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:00.253899 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.260629 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-ab81560c-a197-493e-b5ff-f87512f379af None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:00.310174 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'code': 200} May 25 16:10:00.355035 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.381102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=7']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:00.385607 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.125s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:00.387576 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 00083874-f8bf-4dc1-b252-9739e23df60e, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=00083874-f8bf-4dc1-b252-9739e23df60e,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:00.482693 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.484569 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-574099e9-ab88-45b8-8a66-2a19865ea877 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:00.502339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48874) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:00.510547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-821231136', 'description': 'tempest-secgroup-smoke-821231136 description', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:00.528428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:00.534150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:00.538821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:00.547379 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:00.591652 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=00083874-f8bf-4dc1-b252-9739e23df60e,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:00.594996 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.110s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:00.625645 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.629266 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Sync routers for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:10:00.658392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:00.658596 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:10:00.658596 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:00.658810 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:00.658810 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:00.662193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:00.746313 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.837035 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.842900 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-15df30e7-eeb5-47d4-b0a7-f90cf9b6b90c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:00.875536 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:10:00.900391 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroup': ['ID=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:00.905240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:00.962564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:00.964131 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1792 time: 0.4604492 May 25 16:10:00.967543 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48878) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:00.973408 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:00.979870 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:00.984508 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:00.987629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:00.995701 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:01.148023 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.232684 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.254518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f7a40c65-9a91-404d-9ebd-2ff828ee5ad8 req-d87c9fc5-ba58-406e-8c1d-3ec154899d1e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:01.255733 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f7a40c65-9a91-404d-9ebd-2ff828ee5ad8 req-d87c9fc5-ba58-406e-8c1d-3ec154899d1e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1600833 May 25 16:10:01.312007 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f7a40c65-9a91-404d-9ebd-2ff828ee5ad8 req-3b89077e-98d1-483c-8d0b-aacd8ee12acb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0495415 May 25 16:10:01.313518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.315022 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d71d269f-de23-44ad-8657-06b94100502a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:01.329973 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=6e1e0a6f-00dc-4d3f-9da9-0a160bd7b125,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:01.336798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.021s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:01.359209 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 4, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:10:00Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0', 'name': '', 'network_id': '2df16f06-2058-4b69-828c-2c380456e335', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'mac_address': 'fa:16:3e:b5:7b:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_interface', 'standard_attr_id': 528, 'fixed_ips': [{'subnet_id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:56Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 2, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'subnets': [{'id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:01.364302 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Checking router: dc208968-f643-4678-bf5f-5f1da544823d for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:10:01.364895 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Updating router dc208968-f643-4678-bf5f-5f1da544823d port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:10:01.432264 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.437737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.446545 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.4714086 May 25 16:10:01.446693 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48886) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:01.449137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:01.459666 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:01.470991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:01.478864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:01.486784 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:01.571332 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.696444 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.772948 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 00083874-f8bf-4dc1-b252-9739e23df60e up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:10:01.808459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.816654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-06eee8d7-9cbb-4166-b088-17a6042e91f6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:01.853878 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=90984d75-87f0-4d3a-9a8a-6a20ff02f490,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:01.873964 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:01.875074 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 00083874-f8bf-4dc1-b252-9739e23df60e triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:01.875546 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.944059 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.956841 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Transition to ACTIVE for port object 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:10:01.962550 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:01.969683 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.5229034 May 25 16:10:01.970486 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48890) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:01.976663 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:01.983260 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:01.987220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:01.990641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:02.004388 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:02.053589 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.066932 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-14e9d0d2-cb28-45f6-af13-579e1b1fb576 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:02.088731 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.167257 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.189544 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5078be23-3b00-4703-88fa-77d9aaa5daf4 None None] Pushing event updated for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:02.243292 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5078be23-3b00-4703-88fa-77d9aaa5daf4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.176s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:02.265115 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.272431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3b5ea161-bc94-4130-bbe1-24908ebd1ecd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:02.316657 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=f252ef51-73c2-4c3e-b99f-a5a185a147f8,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:02.326394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:02.374462 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Attempting to bind port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:10:02.375026 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Attempting to bind port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:10:02.375512 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Attempting to bind port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 on network 2df16f06-2058-4b69-828c-2c380456e335 {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:10:02.382091 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 9, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:10:02.382573 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Checking segment: {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:10:02.383127 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.383619 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Bound using segment: {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:10:02.384242 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Bound port: 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1afdd637-4008-4148-a334-b60773d99ad1', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 919, 'network_id': '2df16f06-2058-4b69-828c-2c380456e335'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:10:02.384730 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.401986 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.408086 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48898) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:02.410613 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ff933d48-d653-454f-917e-ae17f9042917 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.4389205 May 25 16:10:02.416903 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:02.421889 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:02.426923 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:02.434926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:02.449999 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:02.580517 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] For port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:10:02.584054 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] For port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:10:02.605404 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afef3518-a2c1-4576-886d-b3753fdabf2e req-c1835759-5e36-441e-8b82-36818dc3b05d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:02.607168 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afef3518-a2c1-4576-886d-b3753fdabf2e req-c1835759-5e36-441e-8b82-36818dc3b05d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1088016 May 25 16:10:02.668989 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afef3518-a2c1-4576-886d-b3753fdabf2e req-b6670989-cb17-4ee5-a2d0-cf3b99ad2ea6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0557399 May 25 16:10:02.745384 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.797173 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.805626 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:02.843672 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.900269 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:10:02.900816 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:09:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:02.968730 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:02.971569 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f379c91f-1679-45f7-9d37-65b0f144178b tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:03.032521 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=91b8ceee-f60e-4580-a650-2d815242fb0a,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:03.049802 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.078s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:03.066470 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5078be23-3b00-4703-88fa-77d9aaa5daf4 None None] Pushing event updated for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:03.072695 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-5078be23-3b00-4703-88fa-77d9aaa5daf4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.267s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:03.123044 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.124530 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.7154365 May 25 16:10:03.127788 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 48906) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:03.139888 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:03.155418 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:03.167469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:03.179193 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:03.193597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:03.310596 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:10:03.310828 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.securitygroups_db [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Project a1e31a264b7949bea6aa47cb7781106b added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:10:03.319361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.436688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.527706 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.532088 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4c8a81eb-a4b6-4cde-8eae-e4a2d577b3a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:03.553946 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=046767eb-d212-4c69-a458-b4b6e5bdc54e,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:03.558289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:03.659303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.666105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 48912) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:03.668178 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.5381277 May 25 16:10:03.672527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'security_group_rule': {'security_group_id': 'dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64', 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:03.685333 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:03.689581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:security_group_rule and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:03.695019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:03.711579 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:03.785122 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.securitygroups_db [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Project a1e31a264b7949bea6aa47cb7781106b added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:10:03.792973 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:03.807211 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 4, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:10:00Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0', 'name': '', 'network_id': '2df16f06-2058-4b69-828c-2c380456e335', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'mac_address': 'fa:16:3e:b5:7b:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_interface', 'standard_attr_id': 528, 'fixed_ips': [{'subnet_id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:56Z', 'updated_at': '2021-05-25T16:10:02Z', 'revision_number': 4, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'subnets': [{'id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:03.815129 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-81e411b2-0b2b-4adc-92cf-2d83105387a8 None None] The sync data for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi is: [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:09:57Z', 'revision_number': 4, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:10:00Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0', 'name': '', 'network_id': '2df16f06-2058-4b69-828c-2c380456e335', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'mac_address': 'fa:16:3e:b5:7b:c8', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_interface', 'standard_attr_id': 528, 'fixed_ips': [{'subnet_id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:56Z', 'updated_at': '2021-05-25T16:10:02Z', 'revision_number': 4, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'subnets': [{'id': '1e6dd188-e084-4f58-b243-e638c41a8cc8', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:10:03.981260 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:04.015996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-43abc606-5e6e-4063-80cd-257842bfca0b req-8a2b17c2-2caa-4aab-bb49-0ca2f756ec28 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:04.023393 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-43abc606-5e6e-4063-80cd-257842bfca0b req-8a2b17c2-2caa-4aab-bb49-0ca2f756ec28 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1926613 May 25 16:10:04.085666 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-43abc606-5e6e-4063-80cd-257842bfca0b req-96117e54-ac48-47ed-b512-94d89006310c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0604231 May 25 16:10:04.099521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:04.101917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:04.133098 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=b38f4371-074f-4aa2-bada-287e46674613,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:04.136109 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.034s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:04.248994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:04.257961 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6514163f-7ae1-4ec6-85bb-f08c2e2df132 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.5825908 May 25 16:10:05.522648 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbcb12e3-a411-4769-8ce5-00b687495f32 req-feea6551-8a1b-4bc8-b1d3-b06fe03bff35 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:05.524851 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbcb12e3-a411-4769-8ce5-00b687495f32 req-feea6551-8a1b-4bc8-b1d3-b06fe03bff35 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.2469811 May 25 16:10:05.595596 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbcb12e3-a411-4769-8ce5-00b687495f32 req-82691797-5d7b-4ca9-b27a-ff6edb1687d8 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0632319 May 25 16:10:05.681455 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e5043688-6ff1-4f8e-b446-c036624f7d1f None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:05.795864 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:10:05.950770 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:10:05.955893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:05.956279 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:06.119857 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:06.165824 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:06.242378 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:06.244429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:06.317860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:06.321171 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:06.811432 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-03253aae-71e2-42bd-b11e-16db793d032d req-763a0311-4a8b-4769-9474-91b33183c77e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:06.812865 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-03253aae-71e2-42bd-b11e-16db793d032d req-763a0311-4a8b-4769-9474-91b33183c77e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1001945 May 25 16:10:06.878992 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-03253aae-71e2-42bd-b11e-16db793d032d req-2756585f-d382-4fa3-8dea-062c29548675 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0553973 May 25 16:10:07.722924 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:10:07.812471 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:07.812985 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:08.098903 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0bdf03f-c87b-4cc0-9260-e5068e26f90c req-7491ded4-82bb-435b-95c1-7e39d6178858 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:08.101899 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0bdf03f-c87b-4cc0-9260-e5068e26f90c req-7491ded4-82bb-435b-95c1-7e39d6178858 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0993261 May 25 16:10:08.144475 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:08.173159 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0bdf03f-c87b-4cc0-9260-e5068e26f90c req-9d0f86c4-a655-4b2b-bfc8-a96f786b7b94 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0611620 May 25 16:10:08.459482 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f12a2421-74f4-4bf6-9150-4b4141151599 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:09.099185 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:09.439774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afe535a3-3252-4341-91ea-b8918fdf20a9 req-dc1484d8-9d89-487d-b552-fb0fc4a8a9ea tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:09.441535 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afe535a3-3252-4341-91ea-b8918fdf20a9 req-dc1484d8-9d89-487d-b552-fb0fc4a8a9ea tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1093204 May 25 16:10:09.501545 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afe535a3-3252-4341-91ea-b8918fdf20a9 req-e6ef0115-aeac-43d5-8ec9-47e51b4d3099 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0526035 May 25 16:10:10.527528 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:10.738987 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a15e6e-28a2-401a-84db-d4e37ac80371 req-891f1d72-1851-4e6d-9d45-513abfc501fe tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:10.740908 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a15e6e-28a2-401a-84db-d4e37ac80371 req-891f1d72-1851-4e6d-9d45-513abfc501fe tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0888247 May 25 16:10:10.823621 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a15e6e-28a2-401a-84db-d4e37ac80371 req-eab8f543-a23c-4cb8-b523-8dbc48417c9c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0755363 May 25 16:10:10.982362 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-5378ad3e-612a-4019-a329-71b3b55f4929 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0856740 May 25 16:10:11.125653 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:11.239785 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-e67a3615-8e99-4302-abc8-a5db5b34a329 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:11.241819 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-e67a3615-8e99-4302-abc8-a5db5b34a329 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.1255407 May 25 16:10:11.547490 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-9c7611dc-2f78-4047-9434-25523fb9cfcd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:11.548848 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-9c7611dc-2f78-4047-9434-25523fb9cfcd service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.2824786 May 25 16:10:11.658972 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-83b52fb5-f686-4b92-b7b4-b78c41fa2bc7 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 1000 time: 0.1033409 May 25 16:10:11.732376 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-d26a00cb-5c60-4aa9-9d24-901d8d3889f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:11.733848 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-d26a00cb-5c60-4aa9-9d24-901d8d3889f6 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.0685046 May 25 16:10:11.820892 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-0925826a-ce79-48e9-95aa-27fb7bde0dde service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:11.822365 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-0925826a-ce79-48e9-95aa-27fb7bde0dde service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0811274 May 25 16:10:11.993137 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-d158c94f-de22-4d95-a53a-556fcb0fc2ec service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1618791 May 25 16:10:12.063322 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5603277c-b7e9-47c0-8685-ba013206331b req-4548d6fc-6296-416d-8f14-c8ce3be6d7ce tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:12.065683 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5603277c-b7e9-47c0-8685-ba013206331b req-4548d6fc-6296-416d-8f14-c8ce3be6d7ce tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1003666 May 25 16:10:12.130643 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5603277c-b7e9-47c0-8685-ba013206331b req-c6444faa-eb39-4c1e-9ff9-e0d5b1035460 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0544429 May 25 16:10:12.169759 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e7dd7e4e-d2fd-4444-84bc-b80114eb57aa req-41abb2cf-8c17-4933-a047-462e21ee9df3 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1697655 May 25 16:10:13.093710 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63ebe91b-f35e-4261-92eb-36694afe603a req-16dfd3ef-113d-4cc4-8aa2-9ac9b6bcdaac service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:13.095589 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-16dfd3ef-113d-4cc4-8aa2-9ac9b6bcdaac service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1099 time: 0.0727558 May 25 16:10:13.268382 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63ebe91b-f35e-4261-92eb-36694afe603a req-e74b5ae6-fc6b-4575-9f67-2046c82cd9cf service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:13.271123 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-e74b5ae6-fc6b-4575-9f67-2046c82cd9cf service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.1534328 May 25 16:10:13.392447 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1a6e63c4-7614-416b-b9c5-1b5d33fa78ab req-eb8ef56e-c0a6-446a-88e2-a435eaacb70a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:13.396673 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1a6e63c4-7614-416b-b9c5-1b5d33fa78ab req-eb8ef56e-c0a6-446a-88e2-a435eaacb70a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1158671 May 25 16:10:13.410691 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-44f795b3-85f1-4e44-a1af-b5672dcf7be7 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 1002 time: 0.1289201 May 25 16:10:13.484331 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1a6e63c4-7614-416b-b9c5-1b5d33fa78ab req-6bcd0009-bb9a-4655-ba61-eeb9a270134d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0782602 May 25 16:10:13.494764 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63ebe91b-f35e-4261-92eb-36694afe603a req-c234cff5-81f7-4363-9ed5-2c2bd7080ddc service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:13.498646 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-c234cff5-81f7-4363-9ed5-2c2bd7080ddc service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0785320 May 25 16:10:13.593221 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-63ebe91b-f35e-4261-92eb-36694afe603a req-2cb1ce7a-59ea-4467-808f-f185613d3f62 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:13.595118 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-2cb1ce7a-59ea-4467-808f-f185613d3f62 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0920184 May 25 16:10:13.742072 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-d131a014-0ed7-433c-a454-c1aaaaeacdc2 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1398015 May 25 16:10:13.890784 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-63ebe91b-f35e-4261-92eb-36694afe603a req-56aed0ba-2f20-4871-94c1-944ea49d1b82 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1411219 May 25 16:10:14.696975 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-db568834-e06c-41d4-b3d8-e5dfe0828b60 req-f0ab193b-83b4-4e88-8f92-a4aa8d908651 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:14.698529 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db568834-e06c-41d4-b3d8-e5dfe0828b60 req-f0ab193b-83b4-4e88-8f92-a4aa8d908651 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.0840681 May 25 16:10:14.752807 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-db568834-e06c-41d4-b3d8-e5dfe0828b60 req-613ff7cf-ca5e-4ee5-bbdb-8f74327c9013 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0469928 May 25 16:10:15.764367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:10:15.892219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:15.925969 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:15.935922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:15.985158 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-34105fbd-76a1-427c-b137-c91a57e788e1 req-5b50729f-67de-49dc-9a34-04514c236986 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:15.987603 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-34105fbd-76a1-427c-b137-c91a57e788e1 req-5b50729f-67de-49dc-9a34-04514c236986 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 946 time: 0.1031244 May 25 16:10:15.992791 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'code': 200} May 25 16:10:16.048178 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:16.050799 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-34105fbd-76a1-427c-b137-c91a57e788e1 req-109bbe40-cf71-40fc-ada1-7cc4968feb4d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0577912 May 25 16:10:16.051459 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:16.128325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=8']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:16.134828 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:17.253557 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-efb05ab8-ba82-47f4-ac5d-a093236bc257 req-ddae88d7-ac15-4b98-adf2-9d094bd4a361 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:17.255065 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-efb05ab8-ba82-47f4-ac5d-a093236bc257 req-ddae88d7-ac15-4b98-adf2-9d094bd4a361 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.0877025 May 25 16:10:17.300262 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-efb05ab8-ba82-47f4-ac5d-a093236bc257 req-8849d175-bd78-40f6-b89a-5d9fc3d146c9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0394483 May 25 16:10:19.769227 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:10:20.273279 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:10:20.316094 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e3f8965-b02b-44e1-b9a2-57a294d9594d req-ab5d8c54-a10f-4cad-9915-cc5aa0ed4b37 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:20.317686 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e3f8965-b02b-44e1-b9a2-57a294d9594d req-ab5d8c54-a10f-4cad-9915-cc5aa0ed4b37 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1255414 May 25 16:10:20.364551 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e3f8965-b02b-44e1-b9a2-57a294d9594d req-7e686c98-4131-4968-a628-85ecd3cb52b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0406322 May 25 16:10:20.691791 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49136) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:20.855127 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-85f5895b-723f-453d-8cd8-49657ddfab8a service neutron] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.1619978 May 25 16:10:21.562684 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-447132f7-ddda-4803-b72e-0f1871294735 req-59e1a10d-699b-4d4f-a593-d9fdb3f50b3f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:21.564290 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-447132f7-ddda-4803-b72e-0f1871294735 req-59e1a10d-699b-4d4f-a593-d9fdb3f50b3f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.0842965 May 25 16:10:21.615090 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-447132f7-ddda-4803-b72e-0f1871294735 req-b23154f1-958a-45e7-8791-3c68e0685114 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0448234 May 25 16:10:22.488758 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-ac8b39cc-e5fa-4557-84da-14fa00f722c3 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:22.490092 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-ac8b39cc-e5fa-4557-84da-14fa00f722c3 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369&tenant_id=690d1dab67d340598496a2b626fceaf2 HTTP/1.1" status: 200 len: 1097 time: 0.0766220 May 25 16:10:22.604264 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-86aa4906-4cea-49c1-b92f-8c8420af682f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:22.606342 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-86aa4906-4cea-49c1-b92f-8c8420af682f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1097 time: 0.0733738 May 25 16:10:22.845779 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-923a2379-a844-486b-b91e-00c6b7c91ab3 req-360025bd-7541-4a4b-bbde-fa02842a2386 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:22.847306 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-923a2379-a844-486b-b91e-00c6b7c91ab3 req-360025bd-7541-4a4b-bbde-fa02842a2386 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1014919 May 25 16:10:22.858422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-50a0a26a-234f-4646-9c86-16738bdcc571 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:22.859905 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-50a0a26a-234f-4646-9c86-16738bdcc571 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.2479866 May 25 16:10:22.895304 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-923a2379-a844-486b-b91e-00c6b7c91ab3 req-e9d09284-e437-4632-84e8-eab01c19bff7 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0417490 May 25 16:10:22.967241 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-91811c9f-480d-4033-b886-0f27ceda1a10 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 1000 time: 0.1004128 May 25 16:10:23.039621 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-1897496d-d62d-40ff-acae-84a30cf95150 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:23.041131 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-1897496d-d62d-40ff-acae-84a30cf95150 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0650210 May 25 16:10:23.123674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-7b70e503-c57c-4a7f-907e-da10bc21d338 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:23.125473 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-7b70e503-c57c-4a7f-907e-da10bc21d338 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0783408 May 25 16:10:23.264260 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-43b0390c-2be9-4b7c-ad98-954dba607f25 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1325486 May 25 16:10:23.409882 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fd291aec-9423-4f89-bbfd-2903a3a342a7 req-c2ec560e-804f-45e9-b91c-93d2090532d3 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1392636 May 25 16:10:23.654002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b5618edb-acbf-4658-94b3-a67008ecd5a9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:10:23.835806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b5618edb-acbf-4658-94b3-a67008ecd5a9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:10:23.837013 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b5618edb-acbf-4658-94b3-a67008ecd5a9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-821231136&fields=id&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 267 time: 0.2281327 May 25 16:10:24.018692 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d9755509-7ffd-4a16-92f6-cb1bcfeeaf16 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:24.020365 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d9755509-7ffd-4a16-92f6-cb1bcfeeaf16 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:10:24.109241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d9755509-7ffd-4a16-92f6-cb1bcfeeaf16 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:10:24.116434 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d9755509-7ffd-4a16-92f6-cb1bcfeeaf16 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.2726078 May 25 16:10:24.148531 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-3d2185da-eea2-47e0-814f-801c6d14b857 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/quotas/a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 375 time: 0.0306935 May 25 16:10:24.220077 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7b52ea40-c3c6-41e5-bb96-43167b3f7004 req-4a880c2b-d6ab-4af5-a7c8-513efbde3c9f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:24.221937 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7b52ea40-c3c6-41e5-bb96-43167b3f7004 req-4a880c2b-d6ab-4af5-a7c8-513efbde3c9f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1776104 May 25 16:10:24.287862 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7b52ea40-c3c6-41e5-bb96-43167b3f7004 req-7c8e8573-ed79-42bf-be3e-c6292c533a82 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0578785 May 25 16:10:24.310532 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-13983f45-64ea-4dbb-840c-9a2c1dd97b4c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1517715 May 25 16:10:24.622928 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-759a02a7-14f9-4531-b24f-6a7aed3eac8b service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.3070941 May 25 16:10:24.821333 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-56e1b186-f055-4429-9e85-640fecaf65f6 service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1953881 May 25 16:10:25.715111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-44ec1ffd-e148-407f-a11f-75205038fe0f req-a6cd478d-d4ae-47e1-bb4e-4bc36c55608b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:25.733141 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-44ec1ffd-e148-407f-a11f-75205038fe0f req-a6cd478d-d4ae-47e1-bb4e-4bc36c55608b tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1505649 May 25 16:10:25.846422 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-44ec1ffd-e148-407f-a11f-75205038fe0f req-16b2a4c0-ac63-4c12-97d0-a1400b1838de tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1021304 May 25 16:10:25.944002 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5a0289f2-a182-4508-a74b-196e4162fbcd req-d5b0be67-5d67-459d-88c7-2e65456281d2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0451553 May 25 16:10:25.972063 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5fe4296d-2016-4c27-b372-fb8e89fb5ff2 req-0abad41f-73cf-4246-a84f-c04e7eb1ce23 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:25.977722 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5fe4296d-2016-4c27-b372-fb8e89fb5ff2 req-0abad41f-73cf-4246-a84f-c04e7eb1ce23 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1715553 May 25 16:10:26.038106 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5fe4296d-2016-4c27-b372-fb8e89fb5ff2 req-98d79408-f5e0-4822-939e-e51e03760af7 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0579536 May 25 16:10:27.070327 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f98574ae-92cb-4693-be98-3b147350d1c5 req-0ec25a26-d739-44cc-a54b-75e2909309f0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:27.072313 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f98574ae-92cb-4693-be98-3b147350d1c5 req-0ec25a26-d739-44cc-a54b-75e2909309f0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1061227 May 25 16:10:27.148003 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ba77cc86-d99d-4415-a570-fae3585725d4 req-d264ad0e-80b8-4c5e-a6ab-94428b542c22 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0543458 May 25 16:10:27.194658 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f98574ae-92cb-4693-be98-3b147350d1c5 req-9d79752b-24aa-480f-9747-7d2a0b392d61 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1168113 May 25 16:10:27.458814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-28c1b562-f595-439f-b316-c19b0a346d40 req-3ac29300-45f8-4c79-a704-b3a55e747cb9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:27.461631 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-28c1b562-f595-439f-b316-c19b0a346d40 req-3ac29300-45f8-4c79-a704-b3a55e747cb9 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1195483 May 25 16:10:27.535725 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-28c1b562-f595-439f-b316-c19b0a346d40 req-54e4c113-635f-4578-843e-836515e43e93 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0665910 May 25 16:10:28.336209 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9ae603cd-bed5-497e-8042-3489c83dfba7 req-5e8d23c7-64bb-4951-ad3c-b9921ce7b62f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0577691 May 25 16:10:28.430927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddda1796-94f8-47fe-b825-057c9684f180 req-c4e7fd4f-d1a6-4d52-a7f6-5b4074ed9d4d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:28.433193 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddda1796-94f8-47fe-b825-057c9684f180 req-c4e7fd4f-d1a6-4d52-a7f6-5b4074ed9d4d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1006651 May 25 16:10:28.487991 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddda1796-94f8-47fe-b825-057c9684f180 req-340ce07f-0f3c-4673-9332-c4b1903eebc1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475404 May 25 16:10:28.796399 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b7db6c77-17c0-4cee-96df-c993d4801167 req-df033938-ba09-41f5-a755-030d55e08ff3 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:28.798486 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b7db6c77-17c0-4cee-96df-c993d4801167 req-df033938-ba09-41f5-a755-030d55e08ff3 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1166141 May 25 16:10:28.852179 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b7db6c77-17c0-4cee-96df-c993d4801167 req-0f2a8f87-1a04-4bcc-9527-6294a3ff104f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0466731 May 25 16:10:29.509757 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ff62ca7d-32b1-45d9-8db1-a7a2e817e23c req-4b9319df-55da-4b84-941f-c0cf571ac588 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0500925 May 25 16:10:29.722046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3498231-6f27-4e8f-b5b7-90fcba717ff3 req-7c692064-bd50-4a04-a385-2bb24891c689 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:29.731286 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a3498231-6f27-4e8f-b5b7-90fcba717ff3 req-7c692064-bd50-4a04-a385-2bb24891c689 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1096761 May 25 16:10:29.811371 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a3498231-6f27-4e8f-b5b7-90fcba717ff3 req-a84b3f1c-5505-45e4-83a6-4de08a1c4c6f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0786464 May 25 16:10:30.115509 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-872c8298-565e-4391-80a6-0bc7f420dc08 req-2cff8c72-8b39-4106-825a-2811fe08f14f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:30.118311 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-872c8298-565e-4391-80a6-0bc7f420dc08 req-2cff8c72-8b39-4106-825a-2811fe08f14f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1098921 May 25 16:10:30.204987 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-872c8298-565e-4391-80a6-0bc7f420dc08 req-1301f233-fac0-47aa-abe0-0b796b331401 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0780315 May 25 16:10:30.741635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f3c1e1ce-cc70-4e50-9ab5-9445ff695a55 req-e7595e07-be2a-4851-acac-6469c24ffede tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0630050 May 25 16:10:31.056532 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-af55816e-a922-4134-a605-04a58ef65ae2 req-2ca34a81-9cee-4ae4-94e5-0b9e100c405d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:31.064239 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-af55816e-a922-4134-a605-04a58ef65ae2 req-2ca34a81-9cee-4ae4-94e5-0b9e100c405d tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1081390 May 25 16:10:31.114393 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-af55816e-a922-4134-a605-04a58ef65ae2 req-d9cff563-72d5-4d6b-8c09-ceb4c11b4aa1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0490832 May 25 16:10:31.469421 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c1efea95-85ca-464c-b7b2-54f5dd4974c6 req-28bbc1dd-273f-4123-b3b8-6cf654259059 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:31.470956 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c1efea95-85ca-464c-b7b2-54f5dd4974c6 req-28bbc1dd-273f-4123-b3b8-6cf654259059 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0980468 May 25 16:10:31.543348 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c1efea95-85ca-464c-b7b2-54f5dd4974c6 req-ce2d4ef1-4de2-4dc8-b26a-6aa8b5c8c90e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0620601 May 25 16:10:31.923297 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-875bd20d-1b98-4c5c-a8a3-25b09cabba85 req-69245749-6352-4872-8976-426624e58549 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0480480 May 25 16:10:32.380282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1f150180-c722-44cd-ab04-37f1a41510aa req-66d77b00-7913-4ead-a94b-0d3da99ae2c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:32.382763 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1f150180-c722-44cd-ab04-37f1a41510aa req-66d77b00-7913-4ead-a94b-0d3da99ae2c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1231141 May 25 16:10:32.436648 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1f150180-c722-44cd-ab04-37f1a41510aa req-209460dc-0e6c-4e06-b489-8ee7ecfc85ef tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0475607 May 25 16:10:32.754754 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-009c27bc-dcba-42ed-b539-0ffeae984eb0 req-de72b6f5-e415-44a4-b19f-5f3a9d04752f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:32.756607 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-009c27bc-dcba-42ed-b539-0ffeae984eb0 req-de72b6f5-e415-44a4-b19f-5f3a9d04752f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0959508 May 25 16:10:32.807443 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-009c27bc-dcba-42ed-b539-0ffeae984eb0 req-f7463fa6-0620-4eb1-8952-7378bc54b01e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0445273 May 25 16:10:33.109242 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-00aa322f-0c8f-46ad-a407-42c777f682e2 req-4d94f841-ae24-44d5-a42e-28478f3eb7c0 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0682375 May 25 16:10:33.742300 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-68558f9a-7eef-4ffc-a5c9-cffdcb650a47 req-37da2dca-b06c-4a1b-8065-ee217ec1e0ce tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:33.744458 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68558f9a-7eef-4ffc-a5c9-cffdcb650a47 req-37da2dca-b06c-4a1b-8065-ee217ec1e0ce tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.1916795 May 25 16:10:33.793279 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-68558f9a-7eef-4ffc-a5c9-cffdcb650a47 req-49fde202-1e7c-42ac-be92-f4bf0c64d7b5 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0417747 May 25 16:10:34.012372 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0708520-0d8b-4e34-8ca3-db900f4b0879 req-89247853-9f44-433b-be4e-e898d98daa85 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:34.013764 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0708520-0d8b-4e34-8ca3-db900f4b0879 req-89247853-9f44-433b-be4e-e898d98daa85 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0865576 May 25 16:10:34.061860 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0708520-0d8b-4e34-8ca3-db900f4b0879 req-e59ee499-8a38-43bc-a61d-e010e6af8f8f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0409021 May 25 16:10:34.286178 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7e5ce3a8-95e7-40de-b9da-455ab90324ae req-8d2e2755-de8b-4d2c-91a8-8dc44f6364de tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0599253 May 25 16:10:35.036637 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-773d4f7d-dfdf-42d2-bf62-1e2861b623c1 req-5b348845-9a0d-4c80-88ab-87ad58cca392 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:35.038241 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-773d4f7d-dfdf-42d2-bf62-1e2861b623c1 req-5b348845-9a0d-4c80-88ab-87ad58cca392 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 944 time: 0.0987480 May 25 16:10:35.102812 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-773d4f7d-dfdf-42d2-bf62-1e2861b623c1 req-bb510e8a-f2ab-4dac-965f-a86e698c92ae tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0462439 May 25 16:10:35.319782 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bdd73b4-d1b5-45cc-ac98-8159b4394ad7 req-7cef9a2b-1b28-4cb4-b24a-90eb385e1a51 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:35.321387 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8bdd73b4-d1b5-45cc-ac98-8159b4394ad7 req-7cef9a2b-1b28-4cb4-b24a-90eb385e1a51 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0930924 May 25 16:10:35.380699 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8bdd73b4-d1b5-45cc-ac98-8159b4394ad7 req-5769c992-913e-4d8f-a423-051c43950e10 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0514922 May 25 16:10:35.473138 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e4c0246a-6bb5-446e-b160-56bb7bd3c892 req-d9d5d774-95b6-48a0-9179-c49506ccf19f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0595961 May 25 16:10:35.676425 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3c32e78a-d9b5-4e33-a386-39d0328ca193 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:35.782633 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:10:35.920061 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:35.934139 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'port_id': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'mac_address': 'fa:16:3e:83:11:68', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 943, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.11'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:10:35.993960 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:36.011270 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:10:36.026593 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:36.161473 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:36.163684 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-339259c7-2041-405e-bdaa-5278dfa3b6f3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:36.242279 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=9']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:36.250602 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.251175 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.251346 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.251602 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.251920 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.252241 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.252561 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.252969 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.253192 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.253618 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.253928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.254267 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:36.258301 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:36.310265 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fb8b2da5-79fd-450e-9623-b9b8620cb1f9 req-1958b940-f41e-4787-87e1-286c09e80631 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:36.311754 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fb8b2da5-79fd-450e-9623-b9b8620cb1f9 req-1958b940-f41e-4787-87e1-286c09e80631 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 945 time: 0.0883460 May 25 16:10:36.359193 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fb8b2da5-79fd-450e-9623-b9b8620cb1f9 req-a25323b9-fc6b-4ed9-8c11-0c0d2a063bee tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0416574 May 25 16:10:36.492381 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['0a449f37-2b2a-4bae-a488-8dcfceef1808'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:10:36.716684 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5c34ec10-cb1f-4a27-b47e-0697c488c323 req-29133ba2-ddf0-4120-b162-e7ad6c458ab3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.0998213 May 25 16:10:36.768924 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5ab39f8d-05ca-43ac-b33a-0e8a3c553d45 req-dc64d5cb-d9f1-498c-9d2d-8ddbb63fbed8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:36.771493 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5ab39f8d-05ca-43ac-b33a-0e8a3c553d45 req-dc64d5cb-d9f1-498c-9d2d-8ddbb63fbed8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.2718251 May 25 16:10:36.824028 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-c3bb2d58-89c6-41a9-8462-41a9315e37c4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:36.835191 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-c3bb2d58-89c6-41a9-8462-41a9315e37c4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.3139637 May 25 16:10:36.859549 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5ab39f8d-05ca-43ac-b33a-0e8a3c553d45 req-4c5724e1-00ea-46e4-8463-5d473e2e2ced tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0791216 May 25 16:10:36.907858 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-46ea34ae-ddb8-48bf-a6d9-ee15b46bbd19 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 379 time: 0.0633583 May 25 16:10:36.916483 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'port': {'device_id': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'admin_state_up': True, 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'security_groups': ['dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64']}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:37.062248 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:37.066354 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:port and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:37.069450 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:37.076266 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:37.195324 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Made reservation on behalf of a1e31a264b7949bea6aa47cb7781106b for: {'port': 1} {{(pid=99544) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:10:37.199392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:37.364615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.20.0.3 (6c61cff6-814b-4af0-8599-e026b705cc35/a4d8980e-d38c-4165-af50-ed2f39dc708c/203fb615-7091-4405-8eb8-c2619684c5d1) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:10:37.559554 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5039e05-bf56-4dcd-b59f-8f4f242310e6 req-a6256fa7-ec79-4f70-bd9a-efcce6b1b0ab tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:37.560211 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5039e05-bf56-4dcd-b59f-8f4f242310e6 req-a6256fa7-ec79-4f70-bd9a-efcce6b1b0ab tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 945 time: 0.0835986 May 25 16:10:37.580417 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:37.603817 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Transition to ACTIVE for port object 203fb615-7091-4405-8eb8-c2619684c5d1 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:10:37.613110 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5039e05-bf56-4dcd-b59f-8f4f242310e6 req-270ac038-2dc0-4ee4-b170-6a82681fc6d4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0469205 May 25 16:10:37.704549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:37.711386 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-45798f37-09b3-45bf-95bf-01888a4ffbcb service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:37.786532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:10:37.801040 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:10:37.801273 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:10:37.805095 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:10:37.871069 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:37.885970 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.887177 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.887708 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.888224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.888827 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.889333 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.889836 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.890532 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.891076 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.891603 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.894864 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.895594 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:37.902834 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.192s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:37.938024 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:37.938717 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:37.976024 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 0a449f37-2b2a-4bae-a488-8dcfceef1808 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:37.976953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.095474 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b420cc52-3b03-4c6c-9062-6f4eee40b2d7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.1823223 May 25 16:10:38.104867 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-29d2e63a-1e9e-4c89-84ca-1555f1d25d03 req-49076900-4a14-4ae2-a0d8-746216910961 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:38.104867 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29d2e63a-1e9e-4c89-84ca-1555f1d25d03 req-49076900-4a14-4ae2-a0d8-746216910961 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 931 time: 0.1838505 May 25 16:10:38.122770 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Request body: {'port': {'device_id': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:38.172217 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-29d2e63a-1e9e-4c89-84ca-1555f1d25d03 req-bb237b4c-cb98-4e27-af17-ea22c2a1fee8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0547271 May 25 16:10:38.179738 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.194790 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-984d9d58-49fb-414a-88df-192d0f668275 req-f37735a2-f4db-4261-878a-e3f2de4bb1df tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:38.196649 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-984d9d58-49fb-414a-88df-192d0f668275 req-f37735a2-f4db-4261-878a-e3f2de4bb1df tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1514373 May 25 16:10:38.203880 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.231378 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:38.243598 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:38.288946 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-984d9d58-49fb-414a-88df-192d0f668275 req-86d8d913-aaee-4ab5-a05b-10479d4311a8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0837464 May 25 16:10:38.309194 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'code': 200} May 25 16:10:38.315759 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ecff2d77-77e3-4bb7-8be1-0bcbe59b8ea4 None None] Provisioning for port 203fb615-7091-4405-8eb8-c2619684c5d1 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:10:38.322162 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-ecff2d77-77e3-4bb7-8be1-0bcbe59b8ea4 None None] Provisioning complete for port 203fb615-7091-4405-8eb8-c2619684c5d1 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:38.322544 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-ecff2d77-77e3-4bb7-8be1-0bcbe59b8ea4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.343424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.357985 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.375307 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-5cdcdb46-c666-4f3c-9e48-793bf748936f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:38.443988 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-ecff2d77-77e3-4bb7-8be1-0bcbe59b8ea4 None None] Port 203fb615-7091-4405-8eb8-c2619684c5d1 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:10:38.466058 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=10']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:38.469683 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:38.538205 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8c5f4156-b313-4852-b369-7bb12a9fa43d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.814499 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9509b99f-0690-4b2f-ad84-63a1b31ee4dc req-5a504c57-1a65-4712-aa33-0be3f4998769 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:38.816090 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9509b99f-0690-4b2f-ad84-63a1b31ee4dc req-5a504c57-1a65-4712-aa33-0be3f4998769 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1100 time: 0.1435688 May 25 16:10:38.852124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 203fb615-7091-4405-8eb8-c2619684c5d1 {{(pid=99544) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:10:38.863435 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.876662 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9509b99f-0690-4b2f-ad84-63a1b31ee4dc req-3ef9234d-937c-45e7-90e3-652ec4131906 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0536535 May 25 16:10:38.882799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Transition to ACTIVE for port object 203fb615-7091-4405-8eb8-c2619684c5d1 will not be triggered until provisioned by entity L2. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:10:38.884571 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9500b18f-3c52-4b44-89d1-3996b9e16826 req-922e84e7-a0eb-449c-aa68-c3c8616cb9fb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:38.886142 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9500b18f-3c52-4b44-89d1-3996b9e16826 req-922e84e7-a0eb-449c-aa68-c3c8616cb9fb tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.1369314 May 25 16:10:38.938437 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9500b18f-3c52-4b44-89d1-3996b9e16826 req-7366fc86-f780-4e43-a74d-1217c98b2147 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0452068 May 25 16:10:38.973553 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:38.978129 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9b89c66-2714-4b73-b304-3b9ffd05f04c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:39.100647 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:39.143638 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Pushing event updated for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:39.153449 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.175s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:39.260298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Attempting to bind port 203fb615-7091-4405-8eb8-c2619684c5d1 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:10:39.260810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Attempting to bind port 203fb615-7091-4405-8eb8-c2619684c5d1 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:10:39.262001 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Attempting to bind port 203fb615-7091-4405-8eb8-c2619684c5d1 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:10:39.267210 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Port 203fb615-7091-4405-8eb8-c2619684c5d1 on network 6c61cff6-814b-4af0-8599-e026b705cc35 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:10:39.267654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Attempting to bind port 203fb615-7091-4405-8eb8-c2619684c5d1 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:10:39.277341 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 10, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:10:39.277750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99544) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:10:39.278122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Bound using segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} {{(pid=99544) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:10:39.278481 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.managers [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Bound port: 203fb615-7091-4405-8eb8-c2619684c5d1, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}}] {{(pid=99544) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:10:39.278829 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:39.383681 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, cleared binding levels {{(pid=99544) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:10:39.387410 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=203fb615-7091-4405-8eb8-c2619684c5d1,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99544) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:10:39.407216 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9e6f1c24-b800-487c-8893-84579fcbd4d6 req-e20d9d1b-e1bd-4c20-9912-1eb7c71995c2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:39.409539 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e6f1c24-b800-487c-8893-84579fcbd4d6 req-e20d9d1b-e1bd-4c20-9912-1eb7c71995c2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0834808 May 25 16:10:39.457602 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9e6f1c24-b800-487c-8893-84579fcbd4d6 req-b9469e21-97d1-4725-a446-2b3dfd137f09 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0428519 May 25 16:10:39.493647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:39.498211 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6fe85a6a-44d9-4514-ac94-534894530130 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:39.560428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:39.561169 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:39.562686 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] 10.222.0.42 "PUT /v2.0/ports/203fb615-7091-4405-8eb8-c2619684c5d1 HTTP/1.1" status: 200 len: 1093 time: 1.4510050 May 25 16:10:39.563310 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:39.597385 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Pushing event updated for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=3']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:39.597385 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:39.650775 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-401156ab-8be3-467d-b74f-d017d428af63 req-b28e88c2-8557-4218-8c3c-385a0ae1c46e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:39.653105 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-401156ab-8be3-467d-b74f-d017d428af63 req-b28e88c2-8557-4218-8c3c-385a0ae1c46e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0991054 May 25 16:10:39.702145 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-401156ab-8be3-467d-b74f-d017d428af63 req-730f6a3f-fe04-459c-b34f-f8dc0162d253 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0431232 May 25 16:10:39.757211 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1', 'status': 'completed', 'code': 200} May 25 16:10:39.879652 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-e169f0e4-5b86-49c5-af06-c578cb4f74b7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:39.881127 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-e169f0e4-5b86-49c5-af06-c578cb4f74b7 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 1096 time: 0.0742457 May 25 16:10:40.050742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-daa5dd44-4300-4bcd-891e-e94b17c8bded service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.052796 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-daa5dd44-4300-4bcd-891e-e94b17c8bded service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=a1e31a264b7949bea6aa47cb7781106b&shared=False HTTP/1.1" status: 200 len: 912 time: 0.1525185 May 25 16:10:40.204872 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d4591c68-5b24-4e83-8757-98d756e19de1 req-e8d96a82-6002-4028-a54b-85f7fa0fca86 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.212310 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d4591c68-5b24-4e83-8757-98d756e19de1 req-e8d96a82-6002-4028-a54b-85f7fa0fca86 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.1429090 May 25 16:10:40.266305 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d4591c68-5b24-4e83-8757-98d756e19de1 req-38d970b1-ef64-4f2e-94f5-10f977d8de28 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0489550 May 25 16:10:40.266447 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-1f23f2c9-0cc0-4005-b9c0-ec2407a24471 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.266447 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-ea4fd2a3-7720-47d1-b6c2-7f7de92e342e req-1f23f2c9-0cc0-4005-b9c0-ec2407a24471 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1985607 May 25 16:10:40.437078 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-91a53f3d-8ebf-4d51-b4cd-9d2a9f3ff676 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.446425 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-91a53f3d-8ebf-4d51-b4cd-9d2a9f3ff676 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 1096 time: 0.0966837 May 25 16:10:40.546221 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-98dc4fee-2db6-4d72-8e88-4b55de32d538 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=203fb615-7091-4405-8eb8-c2619684c5d1 HTTP/1.1" status: 200 len: 217 time: 0.0758703 May 25 16:10:40.595311 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:40.619318 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b9b2e44c-2bf2-4ccc-8d49-3a6aafbfe85a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.621398 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-b9b2e44c-2bf2-4ccc-8d49-3a6aafbfe85a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0674899 May 25 16:10:40.661646 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dfd6f129-fa5b-40a2-83b0-7820e861bf0d req-f160b8bb-86a2-4c82-b377-b115bad7bba9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.663142 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dfd6f129-fa5b-40a2-83b0-7820e861bf0d req-f160b8bb-86a2-4c82-b377-b115bad7bba9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0864766 May 25 16:10:40.716531 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dfd6f129-fa5b-40a2-83b0-7820e861bf0d req-25cd4e6e-6c14-49bd-b1ac-9c39d973f66b tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0453203 May 25 16:10:40.719782 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d72d4689-8e48-4fc1-8bfa-c89883a02e55 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:40.721818 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-d72d4689-8e48-4fc1-8bfa-c89883a02e55 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0941780 May 25 16:10:40.886256 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-2387616d-357e-437c-8aca-8e37536e10ee service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1569302 May 25 16:10:41.019801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-921f34e7-d59b-4a54-a2c5-99df98a33465 req-b6f7b7ac-f077-4f62-bef0-336177bacab0 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:41.021336 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-921f34e7-d59b-4a54-a2c5-99df98a33465 req-b6f7b7ac-f077-4f62-bef0-336177bacab0 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1884501 May 25 16:10:41.084315 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-921f34e7-d59b-4a54-a2c5-99df98a33465 req-417bcc21-5c4e-466a-9db8-3241aada7134 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0544562 May 25 16:10:41.154990 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:41.160327 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-0085daae-cd95-4e98-8b70-630ecd024dfa req-f8f62ca9-6648-4e6f-960b-e6855a1e3a84 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2674034 May 25 16:10:41.471987 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7eef837-6735-44b8-b557-1e56e1949125 req-7a3a47e3-3f48-45e5-842c-e2d96b51945c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:41.473501 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7eef837-6735-44b8-b557-1e56e1949125 req-7a3a47e3-3f48-45e5-842c-e2d96b51945c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.0884881 May 25 16:10:41.518665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7eef837-6735-44b8-b557-1e56e1949125 req-61a6bb0f-d059-4d0b-bd81-970f837fb82f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0390556 May 25 16:10:41.916538 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1beed2f8-05cf-4214-ae7b-d1d714c99d97 req-f0274dc6-6791-418b-af2f-7d0ee2414768 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:41.916538 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1beed2f8-05cf-4214-ae7b-d1d714c99d97 req-f0274dc6-6791-418b-af2f-7d0ee2414768 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0875726 May 25 16:10:41.957328 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1beed2f8-05cf-4214-ae7b-d1d714c99d97 req-fe3a9889-784c-43e7-b5a6-965516ef6fb3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0390382 May 25 16:10:42.274505 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8dffc59a-eff3-4e0f-9593-cca23090aae0 req-2bd96c40-e0d5-400f-9e94-6ec0685b74ff tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:42.276023 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8dffc59a-eff3-4e0f-9593-cca23090aae0 req-2bd96c40-e0d5-400f-9e94-6ec0685b74ff tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0828412 May 25 16:10:42.329112 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8dffc59a-eff3-4e0f-9593-cca23090aae0 req-28a856f5-f1e5-488a-8244-43ea12626ddc tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0459042 May 25 16:10:42.720282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c6190945-3c64-4fdf-a7ab-2e6f25b4af5c req-5c73135a-cc79-47cc-aa1f-5508b7c6445f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:42.721690 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c6190945-3c64-4fdf-a7ab-2e6f25b4af5c req-5c73135a-cc79-47cc-aa1f-5508b7c6445f tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.0860217 May 25 16:10:42.772630 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c6190945-3c64-4fdf-a7ab-2e6f25b4af5c req-3eb770a5-15e4-4587-beda-df11c2a4ac77 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0451570 May 25 16:10:43.148691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d51d98d1-6a58-428b-a841-d1d743ed09ce req-2c147308-75aa-4489-8a91-a4a2f40dd086 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:43.150150 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d51d98d1-6a58-428b-a841-d1d743ed09ce req-2c147308-75aa-4489-8a91-a4a2f40dd086 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0859804 May 25 16:10:43.196015 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d51d98d1-6a58-428b-a841-d1d743ed09ce req-a6f2ae4f-08d1-47ee-a34b-2f62f97725d5 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0399506 May 25 16:10:43.520735 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-278761ac-598c-4a17-b3d4-f3c950247844 req-fde3ba0b-8195-4c50-8f44-6d4680e0e651 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:43.522261 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-278761ac-598c-4a17-b3d4-f3c950247844 req-fde3ba0b-8195-4c50-8f44-6d4680e0e651 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0844047 May 25 16:10:43.571945 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-278761ac-598c-4a17-b3d4-f3c950247844 req-c288501a-5cf8-4c2d-a981-70c4c12840dc tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0433037 May 25 16:10:44.383989 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aa9a0c33-b2d9-43e5-970a-90144be4ff83 req-713c68cf-d7a2-4ed9-9111-41ec9388999f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.385423 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aa9a0c33-b2d9-43e5-970a-90144be4ff83 req-713c68cf-d7a2-4ed9-9111-41ec9388999f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0850742 May 25 16:10:44.430639 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aa9a0c33-b2d9-43e5-970a-90144be4ff83 req-7a8303e6-1d26-452e-91ce-6d82604c5d02 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0385962 May 25 16:10:44.535269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-7aa9d3b2-c9b4-44cb-b1b9-35938adc12f6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.536734 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-7aa9d3b2-c9b4-44cb-b1b9-35938adc12f6 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.0704472 May 25 16:10:44.677222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-7435382b-b72f-4325-b3d3-93bd397fab27 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.678688 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-7435382b-b72f-4325-b3d3-93bd397fab27 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.1348674 May 25 16:10:44.764408 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-466b1fa6-c7a5-40c9-affa-dffb4b82a074 req-6694afe9-f061-4e2e-bdb3-05527ba1e9b8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.765782 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-466b1fa6-c7a5-40c9-affa-dffb4b82a074 req-6694afe9-f061-4e2e-bdb3-05527ba1e9b8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0841000 May 25 16:10:44.787342 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-d34edc8d-c5e2-406b-8606-be6af207446b service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 1000 time: 0.1013825 May 25 16:10:44.817279 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-466b1fa6-c7a5-40c9-affa-dffb4b82a074 req-18684006-2028-476b-adaf-fb1f8abcd7ea tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0452650 May 25 16:10:44.887348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-ec4e57bd-d425-4774-9566-93ad6c3f392a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.888788 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-ec4e57bd-d425-4774-9566-93ad6c3f392a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.0942676 May 25 16:10:44.967780 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-0fd0a338-67c6-4efd-ad10-8df2ff7faa79 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:44.969254 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-0fd0a338-67c6-4efd-ad10-8df2ff7faa79 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0735917 May 25 16:10:45.108132 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-63a85a9a-bfdd-4495-854c-7372032b76d2 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1281476 May 25 16:10:45.242737 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-245d4b05-dd6f-40f5-b53b-944dd6775c19 req-7823939f-1735-451f-9f9f-3c065708e04e service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1275308 May 25 16:10:45.621287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f52b8637-da5a-42df-9fb8-fe5ef74ede46 req-e14d1a5c-8739-436e-b8af-446c33f36e92 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:45.622763 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f52b8637-da5a-42df-9fb8-fe5ef74ede46 req-e14d1a5c-8739-436e-b8af-446c33f36e92 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0830045 May 25 16:10:45.666531 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f52b8637-da5a-42df-9fb8-fe5ef74ede46 req-1846f232-420d-4bda-ac84-d32b8e627833 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0380895 May 25 16:10:46.165723 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9dd489b4-6578-48d6-ac36-fc5a6092d9e1 req-5d1d56bc-5b1f-494c-97fb-a254eb9cd9ee tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:46.167804 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9dd489b4-6578-48d6-ac36-fc5a6092d9e1 req-5d1d56bc-5b1f-494c-97fb-a254eb9cd9ee tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0878959 May 25 16:10:46.214389 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9dd489b4-6578-48d6-ac36-fc5a6092d9e1 req-7d3b5f95-c1e0-418e-abdf-403b20b6c60b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0409222 May 25 16:10:46.861544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-065316f8-384d-4c69-b3c4-d5353c047566 req-8d5a29f0-c8e5-4f64-b64b-b5a114c2d821 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:46.863239 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-065316f8-384d-4c69-b3c4-d5353c047566 req-8d5a29f0-c8e5-4f64-b64b-b5a114c2d821 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0888746 May 25 16:10:46.915573 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-065316f8-384d-4c69-b3c4-d5353c047566 req-c1c1a041-5033-41b6-9d1a-20e6563b2c88 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0450311 May 25 16:10:47.425081 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-87c38ee4-e812-4738-ac67-5940d8bde021 req-1faf8c6f-6208-4adb-ac58-acd4d5eb6cad tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:47.426613 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-87c38ee4-e812-4738-ac67-5940d8bde021 req-1faf8c6f-6208-4adb-ac58-acd4d5eb6cad tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1010129 May 25 16:10:47.476179 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-87c38ee4-e812-4738-ac67-5940d8bde021 req-1b188026-c8d7-4aa7-b8d0-9ede88d9dcf0 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0436153 May 25 16:10:47.899716 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 203fb615-7091-4405-8eb8-c2619684c5d1 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:10:48.046975 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=203fb615-7091-4405-8eb8-c2619684c5d1,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:48.065922 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '203fb615-7091-4405-8eb8-c2619684c5d1', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'port_id': '203fb615-7091-4405-8eb8-c2619684c5d1', 'mac_address': 'fa:16:3e:fe:cd:01', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 902, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.3'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:10:48.125918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-317d4d3b-9308-4899-81ae-2575a28334b9 req-747a7b7a-cc82-45f1-96fd-8b0bae5d2032 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:48.127544 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-317d4d3b-9308-4899-81ae-2575a28334b9 req-747a7b7a-cc82-45f1-96fd-8b0bae5d2032 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 943 time: 0.0959983 May 25 16:10:48.140436 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:48.159807 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 203fb615-7091-4405-8eb8-c2619684c5d1 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:10:48.176537 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-317d4d3b-9308-4899-81ae-2575a28334b9 req-d257a5e3-da97-4d40-b6ff-a5f5a58ee035 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0431807 May 25 16:10:48.178682 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=203fb615-7091-4405-8eb8-c2619684c5d1,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:48.298254 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:48.300083 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:48.368356 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:48.371470 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:48.642101 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['203fb615-7091-4405-8eb8-c2619684c5d1'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:10:48.703601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e6cd75d5-7313-4c72-b4e2-00dc828a83ce req-3b7701e7-7ee8-479c-ad33-3182d65fcb0e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:48.705111 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6cd75d5-7313-4c72-b4e2-00dc828a83ce req-3b7701e7-7ee8-479c-ad33-3182d65fcb0e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1109300 May 25 16:10:48.764116 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e6cd75d5-7313-4c72-b4e2-00dc828a83ce req-6d2b28c7-1adb-4d0d-becb-13725d620534 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0524490 May 25 16:10:49.382283 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9fafa746-22eb-4129-8e18-aa70189ed4f0 req-c12b3ad4-45d2-4d57-b9fc-60d7d7693528 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:49.383766 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9fafa746-22eb-4129-8e18-aa70189ed4f0 req-c12b3ad4-45d2-4d57-b9fc-60d7d7693528 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 944 time: 0.0896020 May 25 16:10:49.441765 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9fafa746-22eb-4129-8e18-aa70189ed4f0 req-19a3b5b4-a0d1-47b9-866f-aa4bfb6807d3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0516765 May 25 16:10:49.911485 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 203fb615-7091-4405-8eb8-c2619684c5d1 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:10:49.993054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bf7baf38-5b78-4e4d-84ed-6c4e1a2109b7 req-e91ddd89-3a1d-47d6-bb67-edeb7f398482 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:49.994668 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf7baf38-5b78-4e4d-84ed-6c4e1a2109b7 req-e91ddd89-3a1d-47d6-bb67-edeb7f398482 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1098688 May 25 16:10:50.046052 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bf7baf38-5b78-4e4d-84ed-6c4e1a2109b7 req-a6c64d8d-3f75-419c-a656-a28a01cb8f3b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0444765 May 25 16:10:50.137988 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 203fb615-7091-4405-8eb8-c2619684c5d1 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:10:50.143855 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 203fb615-7091-4405-8eb8-c2619684c5d1 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:10:50.149556 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:50.271742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:50.300704 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:50.312717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=203fb615-7091-4405-8eb8-c2619684c5d1,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:50.352510 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1', 'code': 200} May 25 16:10:50.396858 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:50.398180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:10:50.426674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:50.566593 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:50.583878 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.157s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:50.668190 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:50.712753 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:50.735131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8f814447-3e0d-4826-abfe-e94f27d7c1e4 req-682b1111-bb3d-49e6-94bb-9a90abedb06c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:50.736728 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8f814447-3e0d-4826-abfe-e94f27d7c1e4 req-682b1111-bb3d-49e6-94bb-9a90abedb06c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.1613283 May 25 16:10:50.773364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5abe6a1a-4f32-45b2-abcc-f2f7df6e9bcc req-de389ca1-bb35-417a-9acc-72b565c1550c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:50.774940 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5abe6a1a-4f32-45b2-abcc-f2f7df6e9bcc req-de389ca1-bb35-417a-9acc-72b565c1550c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1292534 May 25 16:10:50.780478 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-cb47fe26-d026-455d-a0d3-395dfdcc30b2 service neutron] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251&fields=id&fields=resource_request HTTP/1.1" status: 200 len: 255 time: 0.0861878 May 25 16:10:50.800879 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:50.803345 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:50.925561 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=6']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:50.926802 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8f814447-3e0d-4826-abfe-e94f27d7c1e4 req-ee61cd12-ab6e-4b22-9703-c1b884339dc4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1834230 May 25 16:10:50.937180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.134s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:50.963213 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5abe6a1a-4f32-45b2-abcc-f2f7df6e9bcc req-5b34b6c6-a91d-4406-8a72-4e037f12e753 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1756957 May 25 16:10:51.051149 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fdeed28c-0741-4ea0-8b9e-3ce0bc959d7f req-f39a8018-aa1b-40e4-a021-8d6a8f062bba service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:51.052529 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fdeed28c-0741-4ea0-8b9e-3ce0bc959d7f req-f39a8018-aa1b-40e4-a021-8d6a8f062bba service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 1098 time: 0.1478345 May 25 16:10:51.097199 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-fdeed28c-0741-4ea0-8b9e-3ce0bc959d7f req-e5e8bf38-f3f7-41f2-90f5-ba766ca9c1b5 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0379264 May 25 16:10:52.242721 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dc8d82b9-a79f-4ffe-af9d-67232b20d5a0 req-4d61bc18-e2ec-4beb-8c8c-b04ebbc2cea6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:52.244275 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc8d82b9-a79f-4ffe-af9d-67232b20d5a0 req-4d61bc18-e2ec-4beb-8c8c-b04ebbc2cea6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1294761 May 25 16:10:52.252680 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-374d9674-3d3a-4736-8519-1eaa30a808fb req-498e4171-abdc-4a54-955e-7100c3801240 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:52.254125 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-374d9674-3d3a-4736-8519-1eaa30a808fb req-498e4171-abdc-4a54-955e-7100c3801240 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.1900818 May 25 16:10:52.325260 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dc8d82b9-a79f-4ffe-af9d-67232b20d5a0 req-e025766f-92f0-4c70-9c54-1ccc0b0582a2 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0701344 May 25 16:10:52.329997 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-374d9674-3d3a-4736-8519-1eaa30a808fb req-6db49a28-693c-495c-8132-22b77cd2111f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0619431 May 25 16:10:52.356875 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-unplugged', 'status': 'completed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:52.400351 dv-cinder-786384-6-iscsi neutron-server[99546]: WARNING neutron.notifiers.nova [-] Nova event: {'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-unplugged', 'status': 'failed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'code': 422} returned with failed status May 25 16:10:53.590734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8ebfe2a8-3522-4639-8353-b58cc4469c8c req-ebadb4b1-1e3c-4903-8074-85021987e077 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:53.592294 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ebfe2a8-3522-4639-8353-b58cc4469c8c req-ebadb4b1-1e3c-4903-8074-85021987e077 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1472709 May 25 16:10:53.605346 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-70a8b9e7-2897-49d5-87b3-afa8a187a909 req-a5223d34-4c63-46db-8f36-fbd21b7a3ebf tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:53.606856 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70a8b9e7-2897-49d5-87b3-afa8a187a909 req-a5223d34-4c63-46db-8f36-fbd21b7a3ebf tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.1258798 May 25 16:10:53.680857 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8ebfe2a8-3522-4639-8353-b58cc4469c8c req-757c7dcf-7cc1-4af6-8ba1-889f6b0d4bc1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0726197 May 25 16:10:53.687955 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-70a8b9e7-2897-49d5-87b3-afa8a187a909 req-24b5f74f-24f9-4143-abe5-c7b976c7d437 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0663114 May 25 16:10:53.891046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3a6ba11a-2788-4eaa-9cbe-957667c23021 req-60d0f99c-58ed-4c3e-860e-abffdfe69881 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:53.892512 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a6ba11a-2788-4eaa-9cbe-957667c23021 req-60d0f99c-58ed-4c3e-860e-abffdfe69881 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.0920646 May 25 16:10:53.943539 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3a6ba11a-2788-4eaa-9cbe-957667c23021 req-a18d0239-05d8-4f22-a344-729c23821170 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0449085 May 25 16:10:53.972484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49480) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:54.098109 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-445ce896-3091-45b3-b861-8142e9acccf6 tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:54.099700 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-445ce896-3091-45b3-b861-8142e9acccf6 tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 1093 time: 0.1260548 May 25 16:10:54.103396 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49486) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:10:54.108937 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'port_id': '203fb615-7091-4405-8eb8-c2619684c5d1', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address': '172.20.0.3'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:10:54.116781 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:10:54.122038 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:floatingip and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:10:54.125207 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:10:54.132383 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:10:54.275737 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:54.451273 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:54.653334 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.27.2.79 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/19661738-71a9-4fae-b298-282ddccb47a0) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:10:54.869464 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:54.894702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-396c1114-e1b1-4056-bfe0-088d01805396 req-2a9ecbf5-eee0-449c-b7d1-494a120c3449 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:54.896243 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-396c1114-e1b1-4056-bfe0-088d01805396 req-2a9ecbf5-eee0-449c-b7d1-494a120c3449 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0940051 May 25 16:10:54.916542 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cf24522-097c-40ea-984b-616101593c4a req-d6b640c4-e29b-40be-a653-0735c110c3b9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:54.918726 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-d6b640c4-e29b-40be-a653-0735c110c3b9 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251&tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb HTTP/1.1" status: 200 len: 1096 time: 0.1286559 May 25 16:10:54.944887 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-396c1114-e1b1-4056-bfe0-088d01805396 req-26edb18f-9d28-4876-b27a-f15aba7e2291 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0412185 May 25 16:10:55.030559 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:55.035947 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0ef29a5a-abb2-4408-b6cd-f2389c07cb9b service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:55.229674 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cf24522-097c-40ea-984b-616101593c4a req-1d425571-26f9-40f3-9696-36c185b48e68 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:55.231272 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-1d425571-26f9-40f3-9696-36c185b48e68 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1096 time: 0.2676334 May 25 16:10:55.233175 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:10:55.236326 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:10:55.258641 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:10:55.340856 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=19661738-71a9-4fae-b298-282ddccb47a0,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:55.352332 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.316s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:55.482617 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cf24522-097c-40ea-984b-616101593c4a req-faa1ecb3-73e6-42db-a70b-2db51a45eee5 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:55.484088 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-faa1ecb3-73e6-42db-a70b-2db51a45eee5 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.2455921 May 25 16:10:55.696663 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-64df74e8-a902-44b4-a3e5-da7ee912352f service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 998 time: 0.1786535 May 25 16:10:55.854829 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cf24522-097c-40ea-984b-616101593c4a req-f8687aec-5cc3-43f5-8db8-ca5d30a984ad service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:55.856209 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-f8687aec-5cc3-43f5-8db8-ca5d30a984ad service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.1501920 May 25 16:10:55.982293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8cf24522-097c-40ea-984b-616101593c4a req-1ca89093-ee30-43c5-9ee3-c7857fb3c299 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:55.983836 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-1ca89093-ee30-43c5-9ee3-c7857fb3c299 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1216841 May 25 16:10:56.122343 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:56.153426 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-159a0773-194d-4708-b6d2-bd5c9d970141 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1611888 May 25 16:10:56.201608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bde19b78-5c57-4268-ab68-e1bd1a6e3aa9 req-2cce6197-aa63-41b6-a2d8-3554ea5e0492 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:56.204386 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bde19b78-5c57-4268-ab68-e1bd1a6e3aa9 req-2cce6197-aa63-41b6-a2d8-3554ea5e0492 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1230028 May 25 16:10:56.253969 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bde19b78-5c57-4268-ab68-e1bd1a6e3aa9 req-ce58f909-8682-4879-bb50-4a8635926ff4 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0425701 May 25 16:10:56.321245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:56.390714 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-8cf24522-097c-40ea-984b-616101593c4a req-c50cc025-d167-434b-b9c2-f1b5873163f8 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2308307 May 25 16:10:56.568364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:56.663925 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:56.669289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9d4fbd9d-772f-468e-bbeb-d5a6db62c3c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:56.780096 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=19661738-71a9-4fae-b298-282ddccb47a0,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:56.781701 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-148f2533-ca8a-4dbc-90c0-32502090c9f3 req-a3630bce-7df5-4bec-96db-1195ae42120e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:56.783322 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-148f2533-ca8a-4dbc-90c0-32502090c9f3 req-a3630bce-7df5-4bec-96db-1195ae42120e tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1001909 May 25 16:10:56.784824 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:10:56.793955 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:56.835997 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-148f2533-ca8a-4dbc-90c0-32502090c9f3 req-5957318f-b78a-480c-b8fa-617bf4a5e95c tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0448856 May 25 16:10:57.080914 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP 391d4545-674a-434a-9d8e-d2f524b79a84 associated. External IP: 172.27.2.79, port: 203fb615-7091-4405-8eb8-c2619684c5d1. May 25 16:10:57.268737 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:10:57.439484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:10:57.450300 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-49d74ae6-c6f7-440b-bbf1-dd2d4343159b req-3fe9a1c6-4985-48aa-b9c1-5e56e27347cb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:57.452328 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-49d74ae6-c6f7-440b-bbf1-dd2d4343159b req-3fe9a1c6-4985-48aa-b9c1-5e56e27347cb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0875108 May 25 16:10:57.503327 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-49d74ae6-c6f7-440b-bbf1-dd2d4343159b req-8ca60c25-fbfb-46cb-8106-43365db83dc6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0446990 May 25 16:10:57.630604 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:10:57.640602 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:10:57.651506 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:10:57.652224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:10:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:57.720836 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:57.788409 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 3.6839240 May 25 16:10:57.789158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:10:57.852403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:10:57.859808 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1', 'status': 'completed', 'code': 200} May 25 16:10:58.003226 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-103264e5-714c-4d18-aeac-dbe70fb180e6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.003914 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-103264e5-714c-4d18-aeac-dbe70fb180e6 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 1098 time: 0.0952189 May 25 16:10:58.053030 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-905c31e0-b190-4d29-9a7d-7b3fd2d413b8 req-a4640ebb-aa0d-407b-ab67-8f794a66d8f4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.054654 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-905c31e0-b190-4d29-9a7d-7b3fd2d413b8 req-a4640ebb-aa0d-407b-ab67-8f794a66d8f4 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1003330 May 25 16:10:58.119359 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-905c31e0-b190-4d29-9a7d-7b3fd2d413b8 req-0aa9ab26-81ef-4663-b13d-6be55f8d6878 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0578918 May 25 16:10:58.160163 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-60dc369e-60b1-47ee-afbb-17630c36d095 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.162457 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-60dc369e-60b1-47ee-afbb-17630c36d095 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1501224 May 25 16:10:58.242513 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '391d4545-674a-434a-9d8e-d2f524b79a84', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.79', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '203fb615-7091-4405-8eb8-c2619684c5d1', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 540, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:fe:cd:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:10:55Z', 'updated_at': '2021-05-25T16:10:55Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:58.247218 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:10:58.256043 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:10:58.256544 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:10:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:58.341142 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-79eca58b-ff56-4c3e-89e7-ce341380392c service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.3&port_id=203fb615-7091-4405-8eb8-c2619684c5d1 HTTP/1.1" status: 200 len: 998 time: 0.1706629 May 25 16:10:58.409057 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-d40327ea-85ce-40a7-b1cb-2db3a0596135 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.412102 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-d40327ea-85ce-40a7-b1cb-2db3a0596135 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0623684 May 25 16:10:58.463357 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:10:58.499644 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-1d208675-6df7-4ff2-88d6-35c35cad3a7b service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.501815 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-1d208675-6df7-4ff2-88d6-35c35cad3a7b service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0835531 May 25 16:10:58.694835 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-273c2a8e-0810-4ca4-8be4-d6170ae565e4 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1747739 May 25 16:10:58.788364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0808f9cc-1038-42fd-bd17-0a439ffbe3a9 req-21f9f17b-4a2d-499f-bf65-868f78e07972 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.789830 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0808f9cc-1038-42fd-bd17-0a439ffbe3a9 req-21f9f17b-4a2d-499f-bf65-868f78e07972 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1582274 May 25 16:10:58.803641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-bf4c899e-b564-4ac2-96e2-e6a4942297af tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:58.805381 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-bf4c899e-b564-4ac2-96e2-e6a4942297af tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.1367342 May 25 16:10:58.900887 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '391d4545-674a-434a-9d8e-d2f524b79a84', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.79', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '203fb615-7091-4405-8eb8-c2619684c5d1', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 540, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:fe:cd:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:10:55Z', 'updated_at': '2021-05-25T16:10:55Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:10:58.906403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9132ae0b-2d83-448a-ab51-a42272f53edc None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '391d4545-674a-434a-9d8e-d2f524b79a84', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.79', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '203fb615-7091-4405-8eb8-c2619684c5d1', 'fixed_ip_address': '172.20.0.3', 'status': 'DOWN', 'standard_attr_id': 540, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:fe:cd:01', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:10:55Z', 'updated_at': '2021-05-25T16:10:55Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:10:58.922072 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0808f9cc-1038-42fd-bd17-0a439ffbe3a9 req-ad6d3141-72be-4c29-a16e-0a775107056f tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1156173 May 25 16:10:58.960741 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-313489ab-e1e6-4f5b-8c7c-94fe9b991e55 req-db6b11e2-df2d-42a2-b499-ef94738589ab service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2597833 May 25 16:10:59.005887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Deleting port 21c620fe-0dda-41cd-902b-278471e31c3e {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:10:59.006247 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:59.134280 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-0ae65d8b-e30d-4754-b5c3-ce4c283a6c0f None None] New status for floating IP 391d4545-674a-434a-9d8e-d2f524b79a84: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:10:59.153746 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] For port 21c620fe-0dda-41cd-902b-278471e31c3e, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=21c620fe-0dda-41cd-902b-278471e31c3e,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:10:59.154103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:59.617279 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:59.617648 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:59.617964 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Floating IP 2c3aee2e-3b67-4d25-b646-8b9cdeff04b2 disassociated. External IP: 172.27.2.63, port: 21c620fe-0dda-41cd-902b-278471e31c3e. May 25 16:10:59.618307 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Calling delete_port for 21c620fe-0dda-41cd-902b-278471e31c3e owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:10:59.628272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8e697cf1-6b92-4df1-a41c-ce2a03127b68 req-23b93897-42f4-4c98-ae93-385ba5bb00b6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:10:59.629844 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e697cf1-6b92-4df1-a41c-ce2a03127b68 req-23b93897-42f4-4c98-ae93-385ba5bb00b6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 945 time: 0.3646815 May 25 16:10:59.698732 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8e697cf1-6b92-4df1-a41c-ce2a03127b68 req-2c8fb871-8ba5-4cae-be39-8e69618741f6 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups?id=3594719b-1bbe-47d2-8830-e02def651c58&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0611372 May 25 16:10:59.875450 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:10:59.881465 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-efab3974-b747-4414-99c3-71c4036e31d7 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:10:59.927800 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'Port': ['ID=21c620fe-0dda-41cd-902b-278471e31c3e,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:10:59.939234 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.939603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.939963 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.940500 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.940857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.941204 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.941561 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.941905 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.942339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.942684 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.943186 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.943497 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:10:59.954860 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:00.195657 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:11:00.275402 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0ae7c6f1-34b2-4520-aec3-33cc29954bb3 req-3af8a5c5-f588-4780-be83-7c8335fd5970 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:00.277081 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ae7c6f1-34b2-4520-aec3-33cc29954bb3 req-3af8a5c5-f588-4780-be83-7c8335fd5970 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.2377675 May 25 16:11:00.417460 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0ae7c6f1-34b2-4520-aec3-33cc29954bb3 req-b539ae98-eb2b-4206-b006-3865aaf218c1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.1317086 May 25 16:11:00.438947 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:11:00.449958 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:11:00.460606 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:00.461124 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:10:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:00.521801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] No DVR routers for this DVR port 21c620fe-0dda-41cd-902b-278471e31c3e on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:00.524495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:00.527254 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'name': 'network-vif-deleted', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:11:00.539956 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fbd94fdb-ac78-4a7a-bc7f-2e32b0e6aad4 req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.42 "DELETE /v2.0/ports/21c620fe-0dda-41cd-902b-278471e31c3e HTTP/1.1" status: 204 len: 173 time: 1.7097056 May 25 16:11:00.579088 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'b9681fa5-e9c1-41e8-a026-c970c6de830e', 'name': 'network-vif-deleted', 'tag': '21c620fe-0dda-41cd-902b-278471e31c3e', 'status': 'completed', 'code': 200} May 25 16:11:00.737118 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:00.868166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-6ccb4ec0-4e7f-4853-a198-edae138bd8d8 req-75601dbd-1409-4de8-9b65-739a31d00dd3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Synchronizing usage tracker for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:11:00.955976 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-6ccb4ec0-4e7f-4853-a198-edae138bd8d8 req-75601dbd-1409-4de8-9b65-739a31d00dd3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Unset dirty status for tenant:06dda51337ff4f03a0a16af8eaf39a13 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:11:00.957448 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6ccb4ec0-4e7f-4853-a198-edae138bd8d8 req-75601dbd-1409-4de8-9b65-739a31d00dd3 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/ports?device_id=b9681fa5-e9c1-41e8-a026-c970c6de830e HTTP/1.1" status: 200 len: 210 time: 0.1391976 May 25 16:11:01.100140 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:01.104660 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:11:01.113248 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:01.113896 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:10:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:01.301203 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:01.689244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77b4c522-faf8-4e44-b42b-eaa9d0631ae8 req-7bf81314-c124-4087-b604-1ca3eec838e1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:01.690954 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77b4c522-faf8-4e44-b42b-eaa9d0631ae8 req-7bf81314-c124-4087-b604-1ca3eec838e1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.1435330 May 25 16:11:01.703257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:01.714542 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6ac9d0a9-bf00-4f46-ae4b-a9e806e3dce5 None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:03:21Z', 'revision_number': 4, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '709f3425-4852-48b5-bffc-a7829c94b719', 'name': '', 'network_id': '0105f026-2520-418a-a172-829f4b156413', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'mac_address': 'fa:16:3e:12:44:64', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_interface', 'standard_attr_id': 113, 'fixed_ips': [{'subnet_id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:19Z', 'updated_at': '2021-05-25T16:03:33Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'subnets': [{'id': 'c52f7a79-f047-4ca9-9f61-86cb7e470fe5', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:11:01.746835 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77b4c522-faf8-4e44-b42b-eaa9d0631ae8 req-636346a5-e46d-4042-b338-a151481c2c43 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0481894 May 25 16:11:01.956446 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c30e4fa4-a4ee-4610-ba8c-80ecd0b8b2a8 None None] New status for floating IP 2c3aee2e-3b67-4d25-b646-8b9cdeff04b2: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:11:02.055681 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 21c620fe-0dda-41cd-902b-278471e31c3e no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:02.092334 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 21c620fe-0dda-41cd-902b-278471e31c3e {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:02.092680 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 21c620fe-0dda-41cd-902b-278471e31c3e not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:02.151568 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49534) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:02.237961 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:02.269066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:02.435459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:02.439618 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-57765668-5530-4ad6-9417-cb8342c27b4c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:02.455942 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=3594719b-1bbe-47d2-8830-e02def651c58,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:02.458854 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:02.574512 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:02.576191 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "DELETE /v2.0/security-groups/3594719b-1bbe-47d2-8830-e02def651c58 HTTP/1.1" status: 204 len: 168 time: 0.4233806 May 25 16:11:02.578805 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49538) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:02.600369 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.api.v2.resource [None req-14423be6-b684-4738-ac2a-15bd78a8b55f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] show failed (client error): The resource could not be found. May 25 16:11:02.601374 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-14423be6-b684-4738-ac2a-15bd78a8b55f tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "GET /v2.0/security-groups/3594719b-1bbe-47d2-8830-e02def651c58 HTTP/1.1" status: 404 len: 347 time: 0.0214269 May 25 16:11:02.605278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49542) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:02.953936 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Deleting port a222275e-740c-49f0-a5ad-d7f95cbc0fd2 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:02.954392 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:02.967839 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e745f533-ea79-4a9c-98f3-4aa5c9a8c6ff req-253a90f9-f027-42b2-a571-0fd4b94e2b63 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:02.969494 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e745f533-ea79-4a9c-98f3-4aa5c9a8c6ff req-253a90f9-f027-42b2-a571-0fd4b94e2b63 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0923846 May 25 16:11:03.017874 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e745f533-ea79-4a9c-98f3-4aa5c9a8c6ff req-707f5ae3-8a68-4326-a219-5a1a1c30a427 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0423005 May 25 16:11:03.083306 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:03.083546 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:03.212634 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Calling delete_port for a222275e-740c-49f0-a5ad-d7f95cbc0fd2 owned by network:floatingip {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:03.429301 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:03.433750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f157721c-bb80-4057-b8d9-86943307383a tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:03.479194 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Pushing event deleted for resources: {'Port': ['ID=a222275e-740c-49f0-a5ad-d7f95cbc0fd2,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:03.482603 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:03.578364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks [] for floatingip, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:03.674754 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:03.675988 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] 10.222.0.82 "DELETE /v2.0/floatingips/2c3aee2e-3b67-4d25-b646-8b9cdeff04b2 HTTP/1.1" status: 204 len: 168 time: 1.0698032 May 25 16:11:04.062766 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:11:04.206983 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:04.222100 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'port_id': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'mac_address': 'fa:16:3e:2b:e0:91', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 920, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.7'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:11:04.223285 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-359c660a-8da2-4119-b854-8ad91181eb4f req-d2ef87ce-7650-4adb-939a-96f7f6ae3c28 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:04.224868 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-359c660a-8da2-4119-b854-8ad91181eb4f req-d2ef87ce-7650-4adb-939a-96f7f6ae3c28 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 943 time: 0.0946417 May 25 16:11:04.272715 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-359c660a-8da2-4119-b854-8ad91181eb4f req-336b1505-e563-4f97-a212-155fb3455be6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0410686 May 25 16:11:04.287021 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:04.304893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id db215e56-e99f-4ad2-a152-51e2aa8bb9ad {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:11:04.321010 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:04.418303 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:04.420190 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:04.488439 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=7']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:04.491486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.071s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:04.750712 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['db215e56-e99f-4ad2-a152-51e2aa8bb9ad'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:11:05.462856 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-35b3c10a-d8b6-4dae-8911-bc3efbc6752f req-0497f6f8-0f2a-4906-a02b-59a4ee0ca368 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:05.464440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-35b3c10a-d8b6-4dae-8911-bc3efbc6752f req-0497f6f8-0f2a-4906-a02b-59a4ee0ca368 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 944 time: 0.0812275 May 25 16:11:05.516168 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-35b3c10a-d8b6-4dae-8911-bc3efbc6752f req-4b113d8a-5052-4451-855f-df7cfba19f2a tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0457819 May 25 16:11:05.635336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-89d0a82a-43f0-4389-bb0d-dfb0e1f13f6a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:06.078599 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:11:06.189508 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port db215e56-e99f-4ad2-a152-51e2aa8bb9ad triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:11:06.189888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:06.310930 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:06.339473 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:11:06.352024 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:06.385847 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'code': 200} May 25 16:11:06.475043 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:06.476911 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:06.550537 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=8']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:06.552085 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:06.723710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6f4b55cf-0809-4995-9e2b-46a94a438dc3 req-aedef1ca-0a42-4b94-8e5a-12ffd9de21ed tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:06.725407 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6f4b55cf-0809-4995-9e2b-46a94a438dc3 req-aedef1ca-0a42-4b94-8e5a-12ffd9de21ed tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0880575 May 25 16:11:06.777894 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6f4b55cf-0809-4995-9e2b-46a94a438dc3 req-cd64b118-1e7c-41b8-a4d8-ca20d6e38556 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0447962 May 25 16:11:06.963112 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-48b57c80-5880-41ec-9a19-333e5fba89ef req-5d028257-6921-4d7d-88f0-a7677d2149fb service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:06.968397 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-48b57c80-5880-41ec-9a19-333e5fba89ef req-5d028257-6921-4d7d-88f0-a7677d2149fb service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.1783793 May 25 16:11:07.008738 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-48b57c80-5880-41ec-9a19-333e5fba89ef req-3d13033a-55ff-4219-8c12-f189311028b8 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0371890 May 25 16:11:07.973743 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-faf828fa-4070-40a3-b86c-77f2797cc446 req-c1725b95-93d7-4894-8df2-806b6d4e751e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:07.975315 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-faf828fa-4070-40a3-b86c-77f2797cc446 req-c1725b95-93d7-4894-8df2-806b6d4e751e tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0854921 May 25 16:11:08.025438 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-faf828fa-4070-40a3-b86c-77f2797cc446 req-f40e1c72-7851-4a9c-b7e1-b50ed2335b02 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0441520 May 25 16:11:08.167565 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:08.204516 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49588) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:08.428957 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c5af9047-4427-4eb1-9f5d-3664c4c0b421 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:08.975768 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:09.098468 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:09.226266 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3db223ab-f346-4c11-850a-f87b34598c35 req-61478853-3ddc-4bd6-be71-2f126274d63c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:09.227926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3db223ab-f346-4c11-850a-f87b34598c35 req-61478853-3ddc-4bd6-be71-2f126274d63c tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.0908594 May 25 16:11:09.274775 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3db223ab-f346-4c11-850a-f87b34598c35 req-db4bc1b5-dc7e-4797-8469-1f72d63298a3 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0408249 May 25 16:11:09.417548 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Deleting port 709f3425-4852-48b5-bffc-a7829c94b719 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:09.418126 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:09.542765 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] For port 709f3425-4852-48b5-bffc-a7829c94b719, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=709f3425-4852-48b5-bffc-a7829c94b719,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:09.543065 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:09.672629 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Calling delete_port for 709f3425-4852-48b5-bffc-a7829c94b719 owned by network:router_interface {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:10.017793 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:10.022847 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e93b4610-5635-4383-a183-b25ce8bf5298 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:10.100475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Pushing event deleted for resources: {'Port': ['ID=709f3425-4852-48b5-bffc-a7829c94b719,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:10.108471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:10.456976 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:10.508936 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:10.832659 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Router b1fcb15a-38af-4be8-95c4-8843c2278f7c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:11:11.028337 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:11:11.039651 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Sync routers for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:11:11.042972 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3b304e24-c0e1-42a7-8683-b9b2aff03b76 admin admin] 10.222.0.82 "PUT /v2.0/routers/b1fcb15a-38af-4be8-95c4-8843c2278f7c/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.8372610 May 25 16:11:11.045664 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49600) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:11.055179 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:11.055861 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:11.137033 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:11.234346 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks [] for router, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:11.241368 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:11.423058 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:11.539547 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71'}], 'enable_snat': True}, 'gw_port_id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:11:09Z', 'revision_number': 5, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port': {'id': 'e2560d2e-a52a-4dbc-a74c-eda92a478df8', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:3b:bb:43', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 94, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.71', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:07Z', 'updated_at': '2021-05-25T16:03:24Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:11.544271 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Checking router: b1fcb15a-38af-4be8-95c4-8843c2278f7c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:11:11.553076 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Routers {'b1fcb15a-38af-4be8-95c4-8843c2278f7c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:11.553708 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:11.592896 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Deleting port e2560d2e-a52a-4dbc-a74c-eda92a478df8 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:11.593788 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:11.717213 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] Set router b1fcb15a-38af-4be8-95c4-8843c2278f7c gateway port host: None {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:11.731309 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] For port e2560d2e-a52a-4dbc-a74c-eda92a478df8, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=e2560d2e-a52a-4dbc-a74c-eda92a478df8,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:11.731574 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:11.827906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:11:11Z', 'revision_number': 6, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:11.832617 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2e8a7512-eda0-4bb7-9230-7aa2f3462ae9 None None] The sync data for ids ['b1fcb15a-38af-4be8-95c4-8843c2278f7c'] in dv-cinder-786384-6-iscsi is: [{'id': 'b1fcb15a-38af-4be8-95c4-8843c2278f7c', 'name': 'tempest-AttachVolumeTestJSON-1265607496-router', 'tenant_id': '06dda51337ff4f03a0a16af8eaf39a13', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:03:04Z', 'updated_at': '2021-05-25T16:11:11Z', 'revision_number': 6, 'project_id': '06dda51337ff4f03a0a16af8eaf39a13', 'gw_port_host': None}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:11:11.870293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Calling delete_port for e2560d2e-a52a-4dbc-a74c-eda92a478df8 owned by network:router_gateway {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:12.178006 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.183060 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6c01f311-7277-492d-acff-d408fa7c760d tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:12.259110 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Pushing event deleted for resources: {'Port': ['ID=e2560d2e-a52a-4dbc-a74c-eda92a478df8,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:12.266633 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.079s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:12.332567 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.509293 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.512137 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.709755 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.717786 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Fanout notify agent at l3_agent the message router_deleted on router b1fcb15a-38af-4be8-95c4-8843c2278f7c {{(pid=99544) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:11:12.724706 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.725838 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-07731c8a-d3d6-4bfd-8817-0179d50a5e1a admin admin] 10.222.0.82 "DELETE /v2.0/routers/b1fcb15a-38af-4be8-95c4-8843c2278f7c HTTP/1.1" status: 204 len: 168 time: 1.6792419 May 25 16:11:12.729025 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49606) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:12.791476 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Deleting subnet c52f7a79-f047-4ca9-9f61-86cb7e470fe5 {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:11:12.795968 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.856368 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3244650'] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:12.967865 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-aadb9a0f-4224-4a1d-8b6f-704e9a27a0fb service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0719459 May 25 16:11:12.977333 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.078204 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-12689747-7cbf-4348-a021-5f1791f69144 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:13.079620 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-12689747-7cbf-4348-a021-5f1791f69144 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=690d1dab67d340598496a2b626fceaf2&device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 1100 time: 0.0797377 May 25 16:11:13.241303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-83cadfb3-9c58-4b45-af98-f961212748e6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:13.242904 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-83cadfb3-9c58-4b45-af98-f961212748e6 service neutron] 10.222.0.42 "GET /v2.0/networks?id=dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 200 len: 918 time: 0.1433454 May 25 16:11:13.314940 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-3108376', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-3108049'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.319788 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:13.354955 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-4e371fee-9db0-4740-a333-b6a7eb9eff9a service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.11&port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 200 len: 1002 time: 0.1047437 May 25 16:11:13.362978 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.366445 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Pushing event deleted for resources: {'Subnet': ['ID=c52f7a79-f047-4ca9-9f61-86cb7e470fe5,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:13.369523 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:13.370085 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e531ea81-8df2-4147-beb0-f06a3ac04e8b None None] Network 0105f026-2520-418a-a172-829f4b156413 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:11:13.375418 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e63e64aa-0633-4350-b142-5af891f42ccf admin admin] 10.222.0.82 "DELETE /v2.0/subnets/c52f7a79-f047-4ca9-9f61-86cb7e470fe5 HTTP/1.1" status: 204 len: 168 time: 0.6453800 May 25 16:11:13.376166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49610) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:13.432607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-89cb2134-1e06-4177-9708-5bda48a5701e service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:13.435303 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-89cb2134-1e06-4177-9708-5bda48a5701e service neutron] 10.222.0.42 "GET /v2.0/subnets?id=21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 200 len: 856 time: 0.0742686 May 25 16:11:13.502423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.537462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-974cc9ee-bab9-48d4-a8cd-97a3002f8c80 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:13.539094 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-974cc9ee-bab9-48d4-a8cd-97a3002f8c80 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=dc1f952c-66ab-4757-a4c9-eef7df57dea8&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0956149 May 25 16:11:13.581543 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Deleting port 9a1700fb-f69b-45cd-a01d-f45daaa969bb {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:13.581927 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.697625 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] DHCP port deletion for 0105f026-2520-418a-a172-829f4b156413 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:11:13.708579 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] For port 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9a1700fb-f69b-45cd-a01d-f45daaa969bb,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:13.709353 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:13.740276 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-fdffb218-076e-4b00-a151-08c42507041a service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1930883 May 25 16:11:13.791392 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device e2560d2e-a52a-4dbc-a74c-eda92a478df8 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:13.864291 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: e2560d2e-a52a-4dbc-a74c-eda92a478df8 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:13.864837 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device e2560d2e-a52a-4dbc-a74c-eda92a478df8 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:13.865492 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9a1700fb-f69b-45cd-a01d-f45daaa969bb no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:13.972674 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-32379686-27de-4d2c-a9b4-e9b423bd9e09 req-eb328eb1-98c3-4dce-89b1-7d0950d1f915 service neutron] 10.222.0.42 "GET /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2246470 May 25 16:11:13.975787 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Calling delete_port for 9a1700fb-f69b-45cd-a01d-f45daaa969bb owned by network:dhcp {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:14.008631 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Deleting port 9a1700fb-f69b-45cd-a01d-f45daaa969bb {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:14.010278 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.103493 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.108261 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:14.123329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Port 9a1700fb-f69b-45cd-a01d-f45daaa969bb update to DOWN by agent not found {{(pid=99546) update_port_statuses /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2123}} May 25 16:11:14.123787 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 709f3425-4852-48b5-bffc-a7829c94b719 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:14.156009 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Pushing event deleted for resources: {'Port': ['ID=9a1700fb-f69b-45cd-a01d-f45daaa969bb,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:14.159822 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:14.160968 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 709f3425-4852-48b5-bffc-a7829c94b719 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:14.161311 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 709f3425-4852-48b5-bffc-a7829c94b719 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:14.162387 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] No DVR routers for this DVR port 9a1700fb-f69b-45cd-a01d-f45daaa969bb on host dv-cinder-786384-6-iscsi {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:14.183717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] For port 9a1700fb-f69b-45cd-a01d-f45daaa969bb, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9a1700fb-f69b-45cd-a01d-f45daaa969bb,segment=NetworkSegment(4e483d36-603d-4ea7-a03e-3b742a52a6dc),segment_id=4e483d36-603d-4ea7-a03e-3b742a52a6dc)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:14.183717 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.295592 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.310484 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['0105f026-2520-418a-a172-829f4b156413']}} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:14.318849 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Calling delete_port for 9a1700fb-f69b-45cd-a01d-f45daaa969bb owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:14.324457 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '4e483d36-603d-4ea7-a03e-3b742a52a6dc') {'for_net_delete': True} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:14.324750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '4e483d36-603d-4ea7-a03e-3b742a52a6dc') {} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:14.339009 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3244641'] for segment, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.357972 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.365268 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Releasing vlan 932 on physical network physnet1 to pool {{(pid=99544) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:11:14.365671 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-3070910'] for segment, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: Traceback (most recent call last): May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: timer() May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: cb(*args, **kw) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: return func(*args, **kwargs) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: self._notify() May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: self.callback(batched_events) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: event.method(event) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: return f(self, *a, **k) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._get(url).json() May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: return self.request(url, 'GET', **kwargs) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:11:14.380126 dv-cinder-786384-6-iscsi neutron-server[99544]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:11:14.466003 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.469938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:14.514736 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Pushing event deleted for resources: {'Port': ['ID=9a1700fb-f69b-45cd-a01d-f45daaa969bb,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:14.516914 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] No DVR routers for this DVR port 9a1700fb-f69b-45cd-a01d-f45daaa969bb on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:14.518179 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:14.525097 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.529217 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:14.560322 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:14.576837 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Pushing event deleted for resources: {'Network': ['ID=0105f026-2520-418a-a172-829f4b156413,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:14.583081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:14.704245 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] 10.222.0.82 "DELETE /v2.0/networks/0105f026-2520-418a-a172-829f4b156413 HTTP/1.1" status: 204 len: 168 time: 1.3268540 May 25 16:11:15.160704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49618) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:15.229280 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-f8063536-a349-4f88-ad79-7cd20217429c admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=06dda51337ff4f03a0a16af8eaf39a13&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0638206 May 25 16:11:15.231186 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49624) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:15.299528 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:15.330617 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:15.503155 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:15.506982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eb7bc968-52e5-407c-9104-72821a29b6d1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:15.522471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4dec3eef-5a85-4ac9-9518-ae4455aabf44,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:15.526865 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:15.530081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:15.531238 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-84e1329e-9b40-4de3-a348-6ee1ae3616c8 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/4dec3eef-5a85-4ac9-9518-ae4455aabf44 HTTP/1.1" status: 204 len: 168 time: 0.2990634 May 25 16:11:16.422571 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7088b506-ed51-4ffc-82cd-0b3d72faefbc req-0b6225b8-4177-49d1-80e1-05d8278d731a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:16.424234 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7088b506-ed51-4ffc-82cd-0b3d72faefbc req-0b6225b8-4177-49d1-80e1-05d8278d731a tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.0879567 May 25 16:11:16.473789 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7088b506-ed51-4ffc-82cd-0b3d72faefbc req-7aa269f6-10dc-4110-ad34-98bf0f61de1c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0434096 May 25 16:11:17.705190 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-77e87e19-ecdf-496e-af29-b07b016323d1 req-15c2cb0a-f7de-4f12-9783-ce9ffb20eec1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:17.707467 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77e87e19-ecdf-496e-af29-b07b016323d1 req-15c2cb0a-f7de-4f12-9783-ce9ffb20eec1 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.0991673 May 25 16:11:17.781021 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77e87e19-ecdf-496e-af29-b07b016323d1 req-ecd2e02e-d9c7-4859-ae3a-97e2bd9706cc tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups?id=d903c7b0-de69-47ad-94e9-88fec7d7a3f5&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0653424 May 25 16:11:17.977195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-509f6fda-2fa1-4f07-b062-8505b8c42db4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:11:17.979272 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-509f6fda-2fa1-4f07-b062-8505b8c42db4 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 947 time: 0.1035872 May 25 16:11:18.127010 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Deleting port 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:18.127353 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:18.268680 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] For port 0a449f37-2b2a-4bae-a488-8dcfceef1808, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=0a449f37-2b2a-4bae-a488-8dcfceef1808,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:18.269070 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:18.675422 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:18.675801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:18.676120 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Floating IP 772310b5-e9cf-45eb-b710-39e7b132967e disassociated. External IP: 172.27.2.219, port: 0a449f37-2b2a-4bae-a488-8dcfceef1808. May 25 16:11:18.676458 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Calling delete_port for 0a449f37-2b2a-4bae-a488-8dcfceef1808 owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:18.869535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:18.876105 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:18.923466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'Port': ['ID=0a449f37-2b2a-4bae-a488-8dcfceef1808,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:18.926608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:19.059797 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-27fa8742-cf7b-48d0-bd40-8593be5887f9 req-0feab06c-1115-4117-b864-a3ba07da715c tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.1377487 May 25 16:11:19.215194 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:11:19.439642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:11:19.452870 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:11:19.470156 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:19.470800 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:19.537659 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] No DVR routers for this DVR port 0a449f37-2b2a-4bae-a488-8dcfceef1808 on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:19.540254 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:19.545144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-deleted', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:11:19.557445 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b9d88aee-8287-4dd8-b4b1-0d066d1edcdf req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.42 "DELETE /v2.0/ports/0a449f37-2b2a-4bae-a488-8dcfceef1808 HTTP/1.1" status: 204 len: 173 time: 1.5717831 May 25 16:11:19.620931 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'e2f8e138-06c8-4bb4-9988-90e8dde1a369', 'name': 'network-vif-deleted', 'tag': '0a449f37-2b2a-4bae-a488-8dcfceef1808', 'status': 'completed', 'code': 200} May 25 16:11:19.701252 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:20.071164 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:20.075092 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:11:20.083382 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:20.084241 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:20.278733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-d4fa8fc1-c31e-4eed-b8b0-d8bcc07e63db req-f04e7f8d-ba07-47b7-8beb-4de874994103 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Synchronizing usage tracker for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:11:20.314602 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:20.383749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-d4fa8fc1-c31e-4eed-b8b0-d8bcc07e63db req-f04e7f8d-ba07-47b7-8beb-4de874994103 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Unset dirty status for tenant:690d1dab67d340598496a2b626fceaf2 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:11:20.385888 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d4fa8fc1-c31e-4eed-b8b0-d8bcc07e63db req-f04e7f8d-ba07-47b7-8beb-4de874994103 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.1522148 May 25 16:11:20.650579 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:20.654733 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-079fc3aa-034c-412f-92a5-f101123f5879 None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:03:08Z', 'revision_number': 4, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '8e6bec27-45dc-4320-bf1a-16e932839558', 'name': '', 'network_id': 'dc1f952c-66ab-4757-a4c9-eef7df57dea8', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'mac_address': 'fa:16:3e:70:c7:86', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_interface', 'standard_attr_id': 93, 'fixed_ips': [{'subnet_id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:06Z', 'updated_at': '2021-05-25T16:03:19Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'subnets': [{'id': '21c0b655-f786-4f4e-807c-e0313e0d89ae', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:11:20.895945 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3f64f788-830f-4edc-b6f5-7113eab742fc None None] New status for floating IP 772310b5-e9cf-45eb-b710-39e7b132967e: DOWN {{(pid=99546) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:11:21.563863 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2edc352e-7a3f-4984-86f0-e2ac3c14bf7d req-a8575ee0-a340-45da-a057-a7f2ec24a4c9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0442724 May 25 16:11:22.739682 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5d7439de-a298-4a56-9900-5e4b0884fb6e req-c313a723-7d04-42c1-b788-5ae415cd3064 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/ports?device_id=e2f8e138-06c8-4bb4-9988-90e8dde1a369 HTTP/1.1" status: 200 len: 210 time: 0.0475764 May 25 16:11:23.098510 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:23.133708 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 0a449f37-2b2a-4bae-a488-8dcfceef1808 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:23.134058 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 0a449f37-2b2a-4bae-a488-8dcfceef1808 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:23.939242 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49720) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:24.013588 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.045786 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.188032 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.192320 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3f678d25-32f9-4abb-bb33-5e652d171f3b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:24.209362 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=d903c7b0-de69-47ad-94e9-88fec7d7a3f5,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:24.211847 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:24.297385 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.298532 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "DELETE /v2.0/security-groups/d903c7b0-de69-47ad-94e9-88fec7d7a3f5 HTTP/1.1" status: 204 len: 168 time: 0.3582921 May 25 16:11:24.301476 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49728) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:24.318273 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.api.v2.resource [None req-86891f15-0e2b-421b-a9ed-6f187ad2b31e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] show failed (client error): The resource could not be found. May 25 16:11:24.319325 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-86891f15-0e2b-421b-a9ed-6f187ad2b31e tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "GET /v2.0/security-groups/d903c7b0-de69-47ad-94e9-88fec7d7a3f5 HTTP/1.1" status: 404 len: 347 time: 0.0169427 May 25 16:11:24.322762 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49736) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:24.572102 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Deleting port 31116110-ec95-4de5-8fa7-8c5f5faf56da {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:24.572448 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.696051 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.696385 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:24.826569 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Calling delete_port for 31116110-ec95-4de5-8fa7-8c5f5faf56da owned by network:floatingip {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:25.023586 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:25.030414 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9f2615d9-5580-48ae-87dc-7ac8341e7184 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:25.086525 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Pushing event deleted for resources: {'Port': ['ID=31116110-ec95-4de5-8fa7-8c5f5faf56da,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:25.089466 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:25.184820 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks [] for floatingip, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:25.303304 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:25.304489 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-5ce2509b-57eb-40c6-a123-53daadac2e52 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] 10.222.0.82 "DELETE /v2.0/floatingips/772310b5-e9cf-45eb-b710-39e7b132967e HTTP/1.1" status: 204 len: 168 time: 0.9809566 May 25 16:11:27.945496 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49788) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:28.081410 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.098462 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:28.102730 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Usage tracker for resource:security_group_rule and tenant:18b3ebde352c494bbe530540158de1ad is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:28.106339 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:28.113135 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:28.222926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.314203 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.323587 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-43159481-c117-4177-bbf0-cada9255a477 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:28.373581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=051e6225-0211-48a1-852c-62ae59fe6296,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:28.380969 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49798) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:28.381486 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=18b3ebde352c494bbe530540158de1ad&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4343629 May 25 16:11:28.382264 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:28.450651 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.458636 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49804) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:28.479248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.612086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.616889 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ada58eac-eb1f-4cec-86fa-d14fea6d4e16 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:28.631811 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=051e6225-0211-48a1-852c-62ae59fe6296,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:28.635286 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:28.641762 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:28.642959 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/051e6225-0211-48a1-852c-62ae59fe6296 HTTP/1.1" status: 204 len: 168 time: 0.2599294 May 25 16:11:29.222308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:29.694509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Deleting port 8e6bec27-45dc-4320-bf1a-16e932839558 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:29.694951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:29.837967 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] For port 8e6bec27-45dc-4320-bf1a-16e932839558, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=8e6bec27-45dc-4320-bf1a-16e932839558,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:29.838301 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:29.972287 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Calling delete_port for 8e6bec27-45dc-4320-bf1a-16e932839558 owned by network:router_interface {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:30.194593 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:30.199652 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:30.252748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Pushing event deleted for resources: {'Port': ['ID=8e6bec27-45dc-4320-bf1a-16e932839558,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:30.252748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:30.689895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.043702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49860) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:31.176430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Router 1145505b-5e57-4698-b277-66ca05d2520b has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:11:31.216002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.233269 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:31.237650 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Usage tracker for resource:security_group_rule and tenant:add8555d304d429b85fa0060b7d461cf is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:31.240965 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:31.248617 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:31.356911 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.382168 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:11:31.393126 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Sync routers for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:11:31.407303 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-830ff5d5-448c-4881-8c80-ebefc13d5d18 admin admin] 10.222.0.82 "PUT /v2.0/routers/1145505b-5e57-4698-b277-66ca05d2520b/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9473152 May 25 16:11:31.410490 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 49866) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:31.410633 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:31.411519 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:31.433919 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.448749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b9740ac8-6829-4128-950a-cd72a8c82d9b admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:31.508014 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=add8555d304d429b85fa0060b7d461cf&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4632578 May 25 16:11:31.510462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=57498980-7815-4bc9-973f-76899a3fd3bc,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:31.514678 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:31.522603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49870) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:31.623403 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks [] for router, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.630479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.667592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.716127 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:31.817221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.827957 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f8f562a6-facd-4e09-8a1d-e0a7af39347f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:31.846379 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=57498980-7815-4bc9-973f-76899a3fd3bc,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:31.854789 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:31.858814 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/57498980-7815-4bc9-973f-76899a3fd3bc HTTP/1.1" status: 204 len: 168 time: 0.3317463 May 25 16:11:31.858814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:31.864866 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:32.001449 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Deleting port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:32.001810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:32.087528 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28'}], 'enable_snat': True}, 'gw_port_id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:11:30Z', 'revision_number': 5, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port': {'id': '9e92c7c2-9cdb-4242-9af8-0873fad5e72a', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:f9:bf:96', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '1145505b-5e57-4698-b277-66ca05d2520b', 'device_owner': 'network:router_gateway', 'standard_attr_id': 78, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.28', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:03:01Z', 'updated_at': '2021-05-25T16:03:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:32.092670 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Checking router: 1145505b-5e57-4698-b277-66ca05d2520b for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:11:32.103164 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Routers {'1145505b-5e57-4698-b277-66ca05d2520b'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:11:32.103903 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 7, "extensions": [], "floating_ips": 3, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 7, "log_agent_heartbeats": false, "routers": 7},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:11:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['1145505b-5e57-4698-b277-66ca05d2520b'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:32.286152 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] Set router 1145505b-5e57-4698-b277-66ca05d2520b gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:11:32.300772 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] For port 9e92c7c2-9cdb-4242-9af8-0873fad5e72a, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:32.301114 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:32.412201 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:11:31Z', 'revision_number': 6, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:32.417219 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-cd0496de-8419-470f-8d3c-c035b5a3d7ea None None] The sync data for ids ['1145505b-5e57-4698-b277-66ca05d2520b'] in dv-cinder-786384-6-iscsi is: [{'id': '1145505b-5e57-4698-b277-66ca05d2520b', 'name': 'tempest-AttachVolumeShelveTestJSON-1174068749-router', 'tenant_id': '690d1dab67d340598496a2b626fceaf2', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:02:59Z', 'updated_at': '2021-05-25T16:11:31Z', 'revision_number': 6, 'project_id': '690d1dab67d340598496a2b626fceaf2', 'gw_port_host': None}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:11:32.446813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Calling delete_port for 9e92c7c2-9cdb-4242-9af8-0873fad5e72a owned by network:router_gateway {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:32.789869 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:32.796181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-0e40cd64-bc39-4add-9142-317126c9b2f1 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:32.848296 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Pushing event deleted for resources: {'Port': ['ID=9e92c7c2-9cdb-4242-9af8-0873fad5e72a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:32.880974 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.085s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:32.950553 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.159762 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.162840 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.286027 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.286527 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Fanout notify agent at l3_agent the message router_deleted on router 1145505b-5e57-4698-b277-66ca05d2520b {{(pid=99544) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:11:33.304810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.305989 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-42eb3560-793a-43f7-b3c8-93ca8c495e74 admin admin] 10.222.0.82 "DELETE /v2.0/routers/1145505b-5e57-4698-b277-66ca05d2520b HTTP/1.1" status: 204 len: 168 time: 1.8946249 May 25 16:11:33.309328 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49902) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:33.402799 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Deleting subnet 21c0b655-f786-4f4e-807c-e0313e0d89ae {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:11:33.406454 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.465134 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.747952 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:33.767016 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9e92c7c2-9cdb-4242-9af8-0873fad5e72a no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:33.821354 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 9e92c7c2-9cdb-4242-9af8-0873fad5e72a {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:33.822660 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9e92c7c2-9cdb-4242-9af8-0873fad5e72a not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:33.822830 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 8e6bec27-45dc-4320-bf1a-16e932839558 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:33.868795 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 8e6bec27-45dc-4320-bf1a-16e932839558 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:33.869404 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 8e6bec27-45dc-4320-bf1a-16e932839558 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:34.183113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.188478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:34.225240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Pushing event deleted for resources: {'Subnet': ['ID=21c0b655-f786-4f4e-807c-e0313e0d89ae,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:34.230252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:34.230252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.237285 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-f28f23f3-77f5-4782-b9bc-f9b9d56b42db admin admin] 10.222.0.82 "DELETE /v2.0/subnets/21c0b655-f786-4f4e-807c-e0313e0d89ae HTTP/1.1" status: 204 len: 168 time: 0.9268646 May 25 16:11:34.240497 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49918) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:34.241553 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-815647a0-61ff-43e9-bdda-8f9d179a435c None None] Network dc1f952c-66ab-4757-a4c9-eef7df57dea8 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:11:34.270475 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:11:34.384075 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.479601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Deleting port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:34.481182 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.570023 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] DHCP port deletion for dc1f952c-66ab-4757-a4c9-eef7df57dea8 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:11:34.611378 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] For port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:34.611774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.647245 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Deleting port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:11:34.647761 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.744951 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Calling delete_port for 3ec2b131-bb9b-40db-b38d-53666e2fa8ee owned by network:dhcp {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:34.764234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] For port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,segment=NetworkSegment(e08db650-c58e-47d7-8f08-2ac947c7f402),segment_id=e08db650-c58e-47d7-8f08-2ac947c7f402)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:11:34.764790 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.908460 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.912707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:34.933179 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Calling delete_port for 3ec2b131-bb9b-40db-b38d-53666e2fa8ee owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:11:34.961753 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Pushing event deleted for resources: {'Port': ['ID=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:34.965837 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:34.966337 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] No DVR routers for this DVR port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee on host dv-cinder-786384-6-iscsi {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:34.979627 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:34.985093 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:35.029943 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Pushing event deleted for resources: {'Port': ['ID=3ec2b131-bb9b-40db-b38d-53666e2fa8ee,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:35.032339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] No DVR routers for this DVR port 3ec2b131-bb9b-40db-b38d-53666e2fa8ee on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:11:35.034380 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:35.092648 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.107638 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['dc1f952c-66ab-4757-a4c9-eef7df57dea8']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:35.120131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'e08db650-c58e-47d7-8f08-2ac947c7f402') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:35.120426 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'e08db650-c58e-47d7-8f08-2ac947c7f402') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:11:35.134287 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.153931 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.164653 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Releasing vlan 943 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:11:35.165317 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:11:35.191308 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:11:35.406262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.413824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.419424 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:35.446326 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Pushing event deleted for resources: {'Network': ['ID=dc1f952c-66ab-4757-a4c9-eef7df57dea8,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:35.449609 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:35.503669 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-e5c53447-978e-44da-9057-4cbabe37d8ca admin admin] 10.222.0.82 "DELETE /v2.0/networks/dc1f952c-66ab-4757-a4c9-eef7df57dea8 HTTP/1.1" status: 204 len: 168 time: 1.2618709 May 25 16:11:35.617520 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-36484ba2-9b50-49e0-9631-1ee72ee49480 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:35.748726 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 3ec2b131-bb9b-40db-b38d-53666e2fa8ee no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:11:35.790186 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 3ec2b131-bb9b-40db-b38d-53666e2fa8ee {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:11:35.790630 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 3ec2b131-bb9b-40db-b38d-53666e2fa8ee not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:11:35.967927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49942) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:36.021601 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b7faeb32-e552-48da-b898-cba54d00dca3 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=690d1dab67d340598496a2b626fceaf2&name=default HTTP/1.1" status: 200 len: 2975 time: 0.0523078 May 25 16:11:36.024261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 49946) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:36.091759 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:36.121357 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:36.267455 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:36.272248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9ff68855-44bc-46f5-a87c-91ddd57df584 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:36.289171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=145a6cbf-04aa-45cf-a11a-036b86aa1c9c,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:36.294197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:36.299344 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:36.299843 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/145a6cbf-04aa-45cf-a11a-036b86aa1c9c HTTP/1.1" status: 204 len: 168 time: 0.2746837 May 25 16:11:38.188492 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:38.480718 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2a6b89af-d132-44e5-8ee6-85c7b469e69b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:38.856983 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50014) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:39.019348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.035649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:39.042807 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Usage tracker for resource:security_group_rule and tenant:8418ffc4eff7495ca5820c8beb2148f8 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:39.044421 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:39.050548 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:39.114184 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.144348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.234362 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.242170 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6edfd9f5-5115-456c-8bb0-4f5cb6eef6b6 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:39.286569 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=8418ffc4eff7495ca5820c8beb2148f8&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4279883 May 25 16:11:39.287556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=56737765-f13d-483a-878f-3094cc9d3919,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:39.290137 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50020) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:39.293211 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-beba54f6-f566-47bf-b911-b5d9c7d5bf0d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:39.353896 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.380859 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.548976 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.553109 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1a789bed-c413-43ae-8b2f-d40abc390873 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:39.666040 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=56737765-f13d-483a-878f-3094cc9d3919,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:39.671422 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.118s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:39.674961 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:39.676196 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/56737765-f13d-483a-878f-3094cc9d3919 HTTP/1.1" status: 204 len: 168 time: 0.3851249 May 25 16:11:40.387289 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50040) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:40.407175 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:40.424893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:40.429411 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Usage tracker for resource:security_group_rule and tenant:f88b671d2f6f4cffbf78584faffd6fc2 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:40.433967 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:40.440721 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:40.544946 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:40.561229 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:40.657387 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:40.665620 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:40.711577 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f8fb0869-3e0b-437c-83e1-9d204f5ae62c,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:40.717589 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f88b671d2f6f4cffbf78584faffd6fc2&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3279793 May 25 16:11:40.719624 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:40.719854 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50044) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:40.798149 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:40.833980 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:41.010543 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:41.015354 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-2e6a1d52-fc72-4db8-a877-4b301e7755dd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:41.027671 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f8fb0869-3e0b-437c-83e1-9d204f5ae62c,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:41.038621 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.038982 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.039382 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.039739 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.040056 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.040404 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.041232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.041232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.041440 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.041750 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.042095 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.042461 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:11:41.045615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:41.046537 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:41.053920 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a5de04cf-d2cb-462e-8a87-f6c1c8dac7e6 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f8fb0869-3e0b-437c-83e1-9d204f5ae62c HTTP/1.1" status: 204 len: 168 time: 0.3293335 May 25 16:11:41.125860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.117965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50076) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:42.287430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.304773 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:42.308795 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Usage tracker for resource:security_group_rule and tenant:50902b8c3d8049d2bdacdd064b130efe is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:42.311862 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:42.318264 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:42.411980 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.479774 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.489275 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b53a17ed-f354-4f63-86d6-56fd2c847794 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:42.540156 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=50902b8c3d8049d2bdacdd064b130efe&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4167473 May 25 16:11:42.540156 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5d3742eb-0243-457a-9ea2-3684965b14b3,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:42.541377 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50092) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:42.553578 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.064s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:42.631140 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.665245 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.805871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.810589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3847f1af-b859-454a-acf4-79188576f1c0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:42.825381 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5d3742eb-0243-457a-9ea2-3684965b14b3,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:42.829774 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:42.836513 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:42.837755 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b0ed6c8a-d35d-4de3-942c-95473c81a385 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5d3742eb-0243-457a-9ea2-3684965b14b3 HTTP/1.1" status: 204 len: 168 time: 0.2941523 May 25 16:11:43.273947 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50104) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:43.411080 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.428194 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:43.432063 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Usage tracker for resource:security_group_rule and tenant:6405dfb937d94d19b9b1b47e74349640 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:43.434995 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:43.440889 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:43.526614 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.655690 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.665267 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d6664958-d2ed-4c90-8d6a-47534d79ef5d admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:43.707978 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=6405dfb937d94d19b9b1b47e74349640&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4329021 May 25 16:11:43.710168 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d4ac2a74-8426-425a-8742-23a099f91753,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:43.711248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50110) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:43.713919 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-75a9c647-3ec0-43fd-b00a-d8232fb40644 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:43.761569 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50114) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:43.781739 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.782579 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.795728 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:43.800061 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Usage tracker for resource:security_group_rule and tenant:1814322519dc44529fadfe02f941b577 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:43.803601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:43.810440 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:43.813689 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.899906 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.937402 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.941247 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700a0d08-4914-4a07-94ea-cd558c49f622 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:43.954927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d4ac2a74-8426-425a-8742-23a099f91753,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:43.958253 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:43.960449 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.961428 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a998deda-10ee-4081-ba83-6b8b500a7116 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/d4ac2a74-8426-425a-8742-23a099f91753 HTTP/1.1" status: 204 len: 168 time: 0.2488093 May 25 16:11:43.965799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:43.975420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-16e748e2-47b1-4b82-98ba-13e790324ab0 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:44.023586 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=1814322519dc44529fadfe02f941b577&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2608376 May 25 16:11:44.025980 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=ec95b851-2966-42bb-8d88-f912c84b5cb6,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:44.027978 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50120) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:44.032650 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.057s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:44.131890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:44.169092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:44.308973 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:44.312728 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-96d02fd3-e193-4896-9b22-1821006019cb admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:44.324520 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=ec95b851-2966-42bb-8d88-f912c84b5cb6,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:44.336434 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:44.336434 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:44.336434 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/ec95b851-2966-42bb-8d88-f912c84b5cb6 HTTP/1.1" status: 204 len: 168 time: 0.3028247 May 25 16:11:50.865192 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50252) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:50.989297 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.004508 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:51.008355 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Usage tracker for resource:security_group_rule and tenant:f399d8d71f0e4858b875f3215d506403 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:51.011745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:51.017519 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:51.106151 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.183220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.192234 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8478c7e5-00b4-423b-be11-b202b27155ca admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:51.237926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f399d8d71f0e4858b875f3215d506403&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3699741 May 25 16:11:51.239077 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=c4ecba3d-d7ef-43f7-b98a-b619f88f130d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:51.239251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50262) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:51.242602 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7add1bfd-6ca2-4a12-9b11-91ce429e75fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:51.303185 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.332403 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.462699 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.467779 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-628182b0-b11d-4ea0-96e5-1f11c784631f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:51.483361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=c4ecba3d-d7ef-43f7-b98a-b619f88f130d,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:51.485709 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:51.500332 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:51.501699 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ae4f9198-1c6f-457f-8288-321bafd9fc4b admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/c4ecba3d-d7ef-43f7-b98a-b619f88f130d HTTP/1.1" status: 204 len: 168 time: 0.2614446 May 25 16:11:52.160471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50274) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:52.178703 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.194270 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:11:52.198158 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Usage tracker for resource:security_group_rule and tenant:326eefd9cb8343e085f06a049e5c29a4 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:11:52.203949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:11:52.208128 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:11:52.297017 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.353987 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.364859 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f14b4597-059a-401c-a274-5b4d43652d73 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:52.407544 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=326eefd9cb8343e085f06a049e5c29a4&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2459476 May 25 16:11:52.408475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=1dd8d813-3e46-40f8-8dbc-4a3b30045a32,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:52.411218 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50278) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:11:52.411885 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:52.471755 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.497159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.668280 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.672201 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-43e3968a-0fac-40d8-b705-65ae20ab2500 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:11:52.685241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=1dd8d813-3e46-40f8-8dbc-4a3b30045a32,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:11:52.690299 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:11:52.695863 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:11:52.696940 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-fdd4948a-6146-4dc3-99c2-427127b379aa admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/1dd8d813-3e46-40f8-8dbc-4a3b30045a32 HTTP/1.1" status: 204 len: 168 time: 0.2843685 May 25 16:12:00.561329 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50388) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:00.682381 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:00.698756 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:00.703140 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Usage tracker for resource:security_group_rule and tenant:1bc64fd67a9346e384f791db26999584 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:00.706149 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:00.712497 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:00.802297 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:00.903263 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:00.910666 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e27f7805-a34f-4173-938b-6d2f9f47b648 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:00.952541 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=1bc64fd67a9346e384f791db26999584&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3901277 May 25 16:12:00.953361 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5ff9d961-c90b-4991-9c0f-035ae6a01dab,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:00.955970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50394) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:00.956622 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.046s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:01.019301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:01.045244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:01.194124 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:01.201696 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b2e5a1fe-a686-4af4-b29c-a780b2e1fbac admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:01.220308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5ff9d961-c90b-4991-9c0f-035ae6a01dab,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:01.231534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:01.232653 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5ff9d961-c90b-4991-9c0f-035ae6a01dab HTTP/1.1" status: 204 len: 168 time: 0.2760749 May 25 16:12:01.234167 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.234550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.234939 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.235311 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.235884 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.236210 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.236519 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.236878 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.237237 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.237634 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.237939 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.238292 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:01.241838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-eb3cc02a-41e7-4f74-90e7-4a89698aadeb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.040s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:02.003988 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50406) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:02.022622 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.038778 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:02.043272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Usage tracker for resource:security_group_rule and tenant:940ef935d82d4d6091a0d88e3515db9f is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:02.046248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:02.052429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:02.142941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.278119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.285635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2df93cbd-808d-458f-860e-0bd81ab2cacf admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:02.326679 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=940ef935d82d4d6091a0d88e3515db9f&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3217039 May 25 16:12:02.327635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=bb4b7ffe-ffee-4fe9-8d06-8a199a360a2d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:02.331065 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50414) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:02.331624 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b8b56750-a2cb-45ca-8874-b5feef8fbe0c admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:02.395912 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.422754 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.600238 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.604115 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3adc2f87-e134-45fa-886f-d594208e57d9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:02.616491 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=bb4b7ffe-ffee-4fe9-8d06-8a199a360a2d,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:02.620380 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:02.632049 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:02.633346 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e7ea120f-f7d6-4c39-b214-a9074523358f admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/bb4b7ffe-ffee-4fe9-8d06-8a199a360a2d HTTP/1.1" status: 204 len: 168 time: 0.3029258 May 25 16:12:03.463819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50426) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:03.481346 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.496428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:03.500446 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Usage tracker for resource:security_group_rule and tenant:a3bff636f09341b5b8a8b795b322a66b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:03.503514 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:03.509289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:03.600237 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.708506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.716997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4a079c5f-aa5a-41f6-a777-84d0c3f7e283 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:03.760001 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=a3bff636f09341b5b8a8b795b322a66b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2950065 May 25 16:12:03.760797 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9bc091ec-f98a-4a53-9d4b-e0ec7e767f99,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:03.763292 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50430) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:03.764429 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b7668e1b-3b2e-4f0a-8fba-70b85e538325 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:03.834989 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.862405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.994471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:03.999441 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-97fbfc2d-6b37-4760-a4c3-584fb55a2be0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:04.018477 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9bc091ec-f98a-4a53-9d4b-e0ec7e767f99,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:04.023260 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.023s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:04.038562 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:04.039686 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8fb8fc1b-3626-4b96-88b1-a8b213f150ea admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/9bc091ec-f98a-4a53-9d4b-e0ec7e767f99 HTTP/1.1" status: 204 len: 168 time: 0.2752237 May 25 16:12:05.623013 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8a5397e8-fd10-4995-883c-55254a4a9ae2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:08.119814 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:08.434658 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-77114fcb-90db-41e1-a9c7-5e7bfc6a0e03 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:09.165572 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:10.542896 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:11.163485 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:11.272713 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:12:12.973365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50554) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:13.097776 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.207298 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:13.211321 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Usage tracker for resource:security_group_rule and tenant:f49ba82d4d334f5bb7b353721ff70d62 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:13.214189 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:13.220077 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:13.304446 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.417062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.424561 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8ee9744d-e9fc-4aed-a2ef-c5ce57a55a0c admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:13.465729 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=74150bd9-856c-4e67-8585-6908abdf1c3a,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:13.469141 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:13.472122 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f49ba82d4d334f5bb7b353721ff70d62&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4974687 May 25 16:12:13.474813 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50560) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:13.534932 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.562572 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.708669 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.712570 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fab77dbf-fd63-4691-a370-d76c8da04a37 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:13.729631 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=74150bd9-856c-4e67-8585-6908abdf1c3a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:13.732468 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.020s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:13.740630 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:13.741938 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/74150bd9-856c-4e67-8585-6908abdf1c3a HTTP/1.1" status: 204 len: 168 time: 0.2657356 May 25 16:12:14.357822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50610) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:14.375517 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.390471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:14.394359 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Usage tracker for resource:security_group_rule and tenant:f89e40e2c3464edd954e4f138a49045b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:14.397218 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:14.403707 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:14.486622 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.585893 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.592807 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c9a75f13-cc54-4918-abd8-8c85e3113641 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:14.632518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2987f753-3dd6-4ad1-a4ab-c8173182d942,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:14.637260 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=f89e40e2c3464edd954e4f138a49045b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2784019 May 25 16:12:14.638159 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:14.641192 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50614) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:14.709740 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.738521 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.873853 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.877739 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-00f05906-7cc7-4746-8dab-917865c4d4c0 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:14.889801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2987f753-3dd6-4ad1-a4ab-c8173182d942,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:14.893589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:14.897073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:14.898476 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-822b8ebb-3052-44d4-b338-54020592757d admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2987f753-3dd6-4ad1-a4ab-c8173182d942 HTTP/1.1" status: 204 len: 168 time: 0.2561169 May 25 16:12:22.883918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50744) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:23.141910 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50752) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:23.275263 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.291295 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:23.295615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Usage tracker for resource:security_group_rule and tenant:1d27018d32034834a348f9d58bc2a234 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:23.298428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:23.304099 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:23.391378 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.413748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Deleting port 1d6fc6fc-6fb4-4c4a-b162-708178931b62 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:23.414193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.488911 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.497059 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-907ec38f-e6f0-4878-963f-980de8e41f30 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:23.546411 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=1d27018d32034834a348f9d58bc2a234&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4033399 May 25 16:12:23.548819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=f96618c9-3cb8-4ea6-859c-776733fb0a56,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:23.550816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50760) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:23.553030 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-746a6ac3-d6ef-4eb1-996e-663ed3a1d424 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:23.663576 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.663953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.734610 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:23.890536 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Calling delete_port for 1d6fc6fc-6fb4-4c4a-b162-708178931b62 owned by network:floatingip {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:23.922308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.069506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.074481 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-47bada5b-e6b7-4400-a863-eea21279c6d8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:24.089621 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f96618c9-3cb8-4ea6-859c-776733fb0a56,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:24.091890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:24.174487 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.178812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:24.229306 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event deleted for resources: {'Port': ['ID=1d6fc6fc-6fb4-4c4a-b162-708178931b62,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:24.260673 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.082s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:24.316301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.317408 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2a5b7824-4e9d-4296-9732-b1ef57361698 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f96618c9-3cb8-4ea6-859c-776733fb0a56 HTTP/1.1" status: 204 len: 168 time: 0.7652857 May 25 16:12:24.336052 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for floatingip, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.336391 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Floating IP a7837106-14c5-4cd8-9867-49d40c2bd9c1 disassociated (deleted). External IP: 172.27.2.99, port: db215e56-e99f-4ad2-a152-51e2aa8bb9ad. May 25 16:12:24.706592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Router ac04f176-2f32-43dd-b873-426ed50b356d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:12:24.908439 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:12:24.916113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:24.919573 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Sync routers for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:12:24.930636 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:24.931282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:25.020967 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "DELETE /v2.0/floatingips/a7837106-14c5-4cd8-9867-49d40c2bd9c1 HTTP/1.1" status: 204 len: 168 time: 2.1356514 May 25 16:12:25.022908 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:12:25.075006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50774) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:25.094724 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'status': 'completed', 'code': 200} May 25 16:12:25.097460 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.124148 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:25.128657 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Usage tracker for resource:security_group_rule and tenant:5a3487b80b2a4d899ac0b7acf7fa139b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:25.133121 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:25.157836 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:25.186183 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:25.280054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-1f1dce01-9b54-4e85-9f6f-11bccea9e324 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:25.281865 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-1f1dce01-9b54-4e85-9f6f-11bccea9e324 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 1098 time: 0.1497226 May 25 16:12:25.425878 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.474063 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-008d00c3-d659-4c16-a969-823b1e0c276d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:25.476874 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-008d00c3-d659-4c16-a969-823b1e0c276d service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 200 len: 910 time: 0.1879945 May 25 16:12:25.510938 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:25.515368 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Checking router: ac04f176-2f32-43dd-b873-426ed50b356d for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:12:25.524333 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:25.525078 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:25.549320 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-61afdd88-0165-4a23-bec3-ddb883366465 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.7&port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 200 len: 217 time: 0.0660651 May 25 16:12:25.588821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.601664 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0808c5a0-14fd-4b27-be8f-e06f929fb8c8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:25.657128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=9f4672e2-c3c9-4678-a80b-be32d3882283,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:25.660434 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:25.669998 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-640892b9-77a1-4dc6-a330-11b23af2b2bd admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=5a3487b80b2a4d899ac0b7acf7fa139b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.5899861 May 25 16:12:25.669998 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-e2194404-eeab-4c00-972b-8c2ca8e27310 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:25.670420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50778) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:25.670919 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-e2194404-eeab-4c00-972b-8c2ca8e27310 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 200 len: 848 time: 0.1150231 May 25 16:12:25.782900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.786021 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:25.815394 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.981295 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-78a3cf14-3a5b-40cb-ae20-3eaf0be023b7 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:25.982589 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:25.982765 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-78a3cf14-3a5b-40cb-ae20-3eaf0be023b7 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c217a71-5586-47ee-ab1e-21c28b2d40a1&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.3058498 May 25 16:12:25.986949 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5f8beef8-bbb7-4b3a-84fa-c709cf195601 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:26.001643 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=9f4672e2-c3c9-4678-a80b-be32d3882283,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:26.004076 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:26.014251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:26.018468 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/9f4672e2-c3c9-4678-a80b-be32d3882283 HTTP/1.1" status: 204 len: 168 time: 0.3440497 May 25 16:12:26.142207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-20037f6e-19bd-40f5-8115-8a35a5e81d9e req-4e927e09-e218-4742-b255-214362b73af8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:26.142953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-20037f6e-19bd-40f5-8115-8a35a5e81d9e req-4e927e09-e218-4742-b255-214362b73af8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Synchronizing usage tracker for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:12:26.159740 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-e5be0f89-fcc9-4c50-9a22-ccff19e51967 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1665108 May 25 16:12:26.207031 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:26.212594 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-021852df-e929-4049-8efc-fd6cb7307f95 None None] The sync data for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi is: [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:06:35Z', 'revision_number': 4, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'af8ad735-24f2-4b00-92d0-5912edb52f10', 'name': '', 'network_id': '6c217a71-5586-47ee-ab1e-21c28b2d40a1', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'mac_address': 'fa:16:3e:85:68:e4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_interface', 'standard_attr_id': 352, 'fixed_ips': [{'subnet_id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:34Z', 'updated_at': '2021-05-25T16:06:46Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'subnets': [{'id': '2b5e1fe5-3648-40fc-a046-12baa71ed42b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:12:26.252740 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-20037f6e-19bd-40f5-8115-8a35a5e81d9e req-4e927e09-e218-4742-b255-214362b73af8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Unset dirty status for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:12:26.253888 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20037f6e-19bd-40f5-8115-8a35a5e81d9e req-4e927e09-e218-4742-b255-214362b73af8 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.3148577 May 25 16:12:26.337617 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-20037f6e-19bd-40f5-8115-8a35a5e81d9e req-c8d277ef-eeec-45e4-b580-69ffd49de0ae tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0765867 May 25 16:12:26.347358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-212f0f55-42af-4637-8c79-af0a7b9a1e16 req-44fe3e42-e6a7-4b85-9b44-7c53c3ccd978 service neutron] 10.222.0.42 "GET /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1802640 May 25 16:12:26.452714 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-6fef453e-a95c-412a-861b-f8095757d041 None None] New status for floating IP a7837106-14c5-4cd8-9867-49d40c2bd9c1: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:12:26.878550 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50796) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:26.897260 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:26.918939 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:26.923744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Usage tracker for resource:security_group_rule and tenant:85ac0ca44d7c4445ad9871d396ca67b0 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:26.926584 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:26.934643 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:27.039126 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.128853 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.136613 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-5068fbb1-1b14-4a5c-bcd8-51fa8904fa34 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:27.179416 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=85ac0ca44d7c4445ad9871d396ca67b0&name=default HTTP/1.1" status: 200 len: 2979 time: 0.2996342 May 25 16:12:27.180254 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=5e28699e-1ae4-480e-827a-a69f1b192feb,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:27.182024 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50804) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:27.183369 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.047s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:27.242372 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.268490 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.492656 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.502520 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3f1d2083-5280-4111-9f9b-0452292939f3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:27.527747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=5e28699e-1ae4-480e-827a-a69f1b192feb,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:27.530244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:27.543414 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:27.544544 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-658c7b1e-fec2-4d5d-985e-97f2a97c4b67 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/5e28699e-1ae4-480e-827a-a69f1b192feb HTTP/1.1" status: 204 len: 168 time: 0.3615241 May 25 16:12:27.575348 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b8d05f45-39c3-4e2b-bd4c-ff2ed98c993a req-b791f75e-e57c-4957-a52d-ec58536403fe tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:27.577156 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b8d05f45-39c3-4e2b-bd4c-ff2ed98c993a req-b791f75e-e57c-4957-a52d-ec58536403fe tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1221850 May 25 16:12:27.675335 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b8d05f45-39c3-4e2b-bd4c-ff2ed98c993a req-9be18c6b-52b3-4974-a3af-de724dc76c59 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/security-groups?id=503c75ec-7911-4bdd-93c2-bfebd1e51c71&fields=id&fields=name HTTP/1.1" status: 200 len: 313 time: 0.0918469 May 25 16:12:27.703520 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-4f17c384-49fb-4ef7-8b42-3bc38bccac84 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:27.705086 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-4f17c384-49fb-4ef7-8b42-3bc38bccac84 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.42 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 945 time: 0.1020072 May 25 16:12:27.857884 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Deleting port db215e56-e99f-4ad2-a152-51e2aa8bb9ad {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:27.858341 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:28.017012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] For port db215e56-e99f-4ad2-a152-51e2aa8bb9ad, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:28.017012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:28.152242 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Calling delete_port for db215e56-e99f-4ad2-a152-51e2aa8bb9ad owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:28.369284 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:28.373073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-39991ab5-33be-4a94-b72e-bea04080d538 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:28.421997 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event deleted for resources: {'Port': ['ID=db215e56-e99f-4ad2-a152-51e2aa8bb9ad,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:28.424876 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:28.513276 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] No DVR routers for this DVR port db215e56-e99f-4ad2-a152-51e2aa8bb9ad on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:28.516155 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:28.518919 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-deleted', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:12:28.571311 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': 'ee7483db-c622-4701-aa41-45aa235c8251', 'name': 'network-vif-deleted', 'tag': 'db215e56-e99f-4ad2-a152-51e2aa8bb9ad', 'status': 'completed', 'code': 200} May 25 16:12:28.657698 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5a8d946c-631b-4279-ae0b-3dfa5d44f020 req-b47bc22c-ad5a-4b6b-9b07-e95d37410384 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.42 "DELETE /v2.0/ports/db215e56-e99f-4ad2-a152-51e2aa8bb9ad HTTP/1.1" status: 204 len: 173 time: 0.9467375 May 25 16:12:28.853817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-6ef31107-7824-4d30-81ea-10eba47d0077 req-72ce833c-6f98-46ce-a966-b1bcf6a5eb31 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Synchronizing usage tracker for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:12:28.944280 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-6ef31107-7824-4d30-81ea-10eba47d0077 req-72ce833c-6f98-46ce-a966-b1bcf6a5eb31 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Unset dirty status for tenant:5d4b9c0f27af44a9ace8862a2133a0cb on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:12:28.944280 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6ef31107-7824-4d30-81ea-10eba47d0077 req-72ce833c-6f98-46ce-a966-b1bcf6a5eb31 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.1381922 May 25 16:12:29.753339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50862) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:29.908901 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:29.923822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:29.928589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Usage tracker for resource:security_group_rule and tenant:48f511f4b85b4bee94154f5b4695a828 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:29.931435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:29.937790 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:30.023057 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.110151 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e840fdb3-f3b0-46f5-8e34-d7854a6974f6 req-da67a2bc-d02c-4826-afaf-92aac7f806ba tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "GET /v2.0/ports?device_id=ee7483db-c622-4701-aa41-45aa235c8251 HTTP/1.1" status: 200 len: 210 time: 0.0488122 May 25 16:12:30.142171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.149172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-bbed439f-6c99-4c5a-8016-229b3ba5c578 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:30.193645 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=48f511f4b85b4bee94154f5b4695a828&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4394009 May 25 16:12:30.194615 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=4ef750ce-6823-44e4-bfd4-9bb78213cf30,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:30.196774 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50870) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:30.199244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:30.260819 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.287942 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.513951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.517857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-19fed96e-1c8e-436d-b89d-dd3679c511b6 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:30.531222 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=4ef750ce-6823-44e4-bfd4-9bb78213cf30,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:30.533453 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.016s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:30.541282 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:30.545719 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/4ef750ce-6823-44e4-bfd4-9bb78213cf30 HTTP/1.1" status: 204 len: 168 time: 0.3478386 May 25 16:12:30.607344 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:30.642683 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: db215e56-e99f-4ad2-a152-51e2aa8bb9ad {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:30.643101 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device db215e56-e99f-4ad2-a152-51e2aa8bb9ad not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:31.230766 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50890) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:31.306254 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:31.336966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:31.485311 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:31.490428 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-dfadde2c-ff7b-4658-b1ed-5002f0eaf629 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:31.509153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=503c75ec-7911-4bdd-93c2-bfebd1e51c71,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:31.512012 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.022s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:31.667601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:31.668895 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-83811689-8ca5-47b4-9a9d-af4e03d92deb tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] 10.222.0.82 "DELETE /v2.0/security-groups/503c75ec-7911-4bdd-93c2-bfebd1e51c71 HTTP/1.1" status: 204 len: 168 time: 0.4371030 May 25 16:12:31.843844 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50910) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:32.583443 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:33.197026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Deleting port af8ad735-24f2-4b00-92d0-5912edb52f10 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:33.197862 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:33.334785 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] For port af8ad735-24f2-4b00-92d0-5912edb52f10, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=af8ad735-24f2-4b00-92d0-5912edb52f10,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:33.335555 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:33.539947 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Calling delete_port for af8ad735-24f2-4b00-92d0-5912edb52f10 owned by network:router_interface {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:33.797895 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:33.802241 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a0eb8b85-44f5-4cd4-9394-358dffcb5341 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:33.848928 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Pushing event deleted for resources: {'Port': ['ID=af8ad735-24f2-4b00-92d0-5912edb52f10,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:33.852279 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:34.175121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:34.551229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Router ac04f176-2f32-43dd-b873-426ed50b356d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:12:34.850845 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:12:34.860369 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Sync routers for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:12:34.870525 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:34.871204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:34.927132 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-fa298b59-ba9c-41ff-bba8-b8dc060b3231 admin admin] 10.222.0.82 "PUT /v2.0/routers/ac04f176-2f32-43dd-b873-426ed50b356d/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.0820456 May 25 16:12:34.929677 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50942) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:35.062466 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:35.116283 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks [] for router, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:35.316650 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:35.361928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159'}], 'enable_snat': True}, 'gw_port_id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:12:33Z', 'revision_number': 5, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port': {'id': 'bb4cb419-146e-473c-abad-9dffa503b894', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:ec:57:e2', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 351, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.159', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:26Z', 'updated_at': '2021-05-25T16:06:38Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:35.366372 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Checking router: ac04f176-2f32-43dd-b873-426ed50b356d for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:12:35.383134 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Routers {'ac04f176-2f32-43dd-b873-426ed50b356d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:35.383786 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['ac04f176-2f32-43dd-b873-426ed50b356d'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:35.487772 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Deleting port bb4cb419-146e-473c-abad-9dffa503b894 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:35.488257 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:35.575189 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] Set router ac04f176-2f32-43dd-b873-426ed50b356d gateway port host: None {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:35.639744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] For port bb4cb419-146e-473c-abad-9dffa503b894, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bb4cb419-146e-473c-abad-9dffa503b894,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:35.640106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:35.763135 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:12:35Z', 'revision_number': 6, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:35.771757 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87c843fc-3f21-4793-9422-e87e1788cdca None None] The sync data for ids ['ac04f176-2f32-43dd-b873-426ed50b356d'] in dv-cinder-786384-6-iscsi is: [{'id': 'ac04f176-2f32-43dd-b873-426ed50b356d', 'name': 'tempest-TestShelveInstance-1030692280-router', 'tenant_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:22Z', 'updated_at': '2021-05-25T16:12:35Z', 'revision_number': 6, 'project_id': '5d4b9c0f27af44a9ace8862a2133a0cb', 'gw_port_host': None}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:12:35.786252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Calling delete_port for bb4cb419-146e-473c-abad-9dffa503b894 owned by network:router_gateway {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:35.808542 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8825f0fd-4b4e-4ff8-822e-65685792a461 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.014423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.018504 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-52e55283-b637-4187-88c7-95661fc4c594 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:36.068321 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Pushing event deleted for resources: {'Port': ['ID=bb4cb419-146e-473c-abad-9dffa503b894,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:36.095242 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.077s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:36.165474 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.348389 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.350683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.459826 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.460335 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Fanout notify agent at l3_agent the message router_deleted on router ac04f176-2f32-43dd-b873-426ed50b356d {{(pid=99544) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:12:36.472831 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.473929 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] 10.222.0.82 "DELETE /v2.0/routers/ac04f176-2f32-43dd-b873-426ed50b356d HTTP/1.1" status: 204 len: 168 time: 1.5433416 May 25 16:12:36.476598 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 50956) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:36.539106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Deleting subnet 2b5e1fe5-3648-40fc-a046-12baa71ed42b {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:12:36.543410 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.597218 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3244650'] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:36.810294 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.171970 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-3108376', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-3108049'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.176870 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-78e22c92-a15a-4e1a-8bad-8c445f4d3a86 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:37.218798 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=2b5e1fe5-3648-40fc-a046-12baa71ed42b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:37.221880 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.045s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:37.223079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.230731 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-32a5ec13-ec2c-4ed0-8d4c-a168c84ddf3f None None] Network 6c217a71-5586-47ee-ab1e-21c28b2d40a1 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:12:37.234716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50964) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:37.235494 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-ef983dc0-1884-4f74-a53f-e630c6d818d4 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/2b5e1fe5-3648-40fc-a046-12baa71ed42b HTTP/1.1" status: 204 len: 168 time: 0.7570593 May 25 16:12:37.460249 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.541278 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Deleting port b5e082ba-8608-48eb-ba90-ac1afb9263ff {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:37.542176 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.647021 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] DHCP port deletion for 6c217a71-5586-47ee-ab1e-21c28b2d40a1 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:12:37.656318 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] For port b5e082ba-8608-48eb-ba90-ac1afb9263ff, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b5e082ba-8608-48eb-ba90-ac1afb9263ff,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:37.656318 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.722315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Deleting port b5e082ba-8608-48eb-ba90-ac1afb9263ff {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:37.722843 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.788693 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Calling delete_port for b5e082ba-8608-48eb-ba90-ac1afb9263ff owned by network:dhcp {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:37.812203 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device af8ad735-24f2-4b00-92d0-5912edb52f10 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:37.915697 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: af8ad735-24f2-4b00-92d0-5912edb52f10 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:37.916286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device af8ad735-24f2-4b00-92d0-5912edb52f10 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:37.917590 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b5e082ba-8608-48eb-ba90-ac1afb9263ff no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:37.942820 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:37.947373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:37.986423 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: b5e082ba-8608-48eb-ba90-ac1afb9263ff {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:37.986423 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b5e082ba-8608-48eb-ba90-ac1afb9263ff not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:37.986423 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bb4cb419-146e-473c-abad-9dffa503b894 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:38.006255 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Pushing event deleted for resources: {'Port': ['ID=b5e082ba-8608-48eb-ba90-ac1afb9263ff,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:38.012019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.062s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:38.015828 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] No DVR routers for this DVR port b5e082ba-8608-48eb-ba90-ac1afb9263ff on host dv-cinder-786384-6-iscsi {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:38.044111 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: bb4cb419-146e-473c-abad-9dffa503b894 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:38.044674 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bb4cb419-146e-473c-abad-9dffa503b894 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:38.049359 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] For port b5e082ba-8608-48eb-ba90-ac1afb9263ff, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b5e082ba-8608-48eb-ba90-ac1afb9263ff,segment=NetworkSegment(79c416e6-aabc-4638-95ed-e2e21a7b289f),segment_id=79c416e6-aabc-4638-95ed-e2e21a7b289f)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:38.049950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.221736 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.230467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Calling delete_port for b5e082ba-8608-48eb-ba90-ac1afb9263ff owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:38.240364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['6c217a71-5586-47ee-ab1e-21c28b2d40a1']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:38.256108 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '79c416e6-aabc-4638-95ed-e2e21a7b289f') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:38.256495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '79c416e6-aabc-4638-95ed-e2e21a7b289f') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:38.269527 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.285288 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.290351 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.291348 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:38.298590 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Releasing vlan 920 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:12:38.299094 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:12:38.314422 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:12:38.344234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Pushing event deleted for resources: {'Port': ['ID=b5e082ba-8608-48eb-ba90-ac1afb9263ff,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:38.351220 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] No DVR routers for this DVR port b5e082ba-8608-48eb-ba90-ac1afb9263ff on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:38.358371 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.369512 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.370570 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.371022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.371996 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.371996 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.371996 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.372442 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.373046 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.373583 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.373734 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.374059 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.374397 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:38.380173 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.089s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:38.575754 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c3e1a9b5-d202-4a68-b84a-527425079017 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.632639 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.638083 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-700cca12-ef4f-49b5-9e32-95a3bd02f13f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:38.653613 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:38.666394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Pushing event deleted for resources: {'Network': ['ID=6c217a71-5586-47ee-ab1e-21c28b2d40a1,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:38.671294 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:38.769921 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] 10.222.0.82 "DELETE /v2.0/networks/6c217a71-5586-47ee-ab1e-21c28b2d40a1 HTTP/1.1" status: 204 len: 168 time: 1.5331535 May 25 16:12:38.773314 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50972) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:39.168222 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:39.376360 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:39.955889 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Deleting port d69a72d5-419a-489c-b21f-eee55561f332 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:39.956218 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:40.082715 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] For port d69a72d5-419a-489c-b21f-eee55561f332, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d69a72d5-419a-489c-b21f-eee55561f332,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:40.083006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:40.210345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Calling delete_port for d69a72d5-419a-489c-b21f-eee55561f332 owned by network:router_interface {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:40.534008 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:40.563189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:40.568504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8e4c8cb0-90b6-48cf-9666-e777791b1942 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:40.649385 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Pushing event deleted for resources: {'Port': ['ID=d69a72d5-419a-489c-b21f-eee55561f332,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:40.651943 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.084s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:40.889244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:41.140761 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:41.478201 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Router 2fc03949-9293-45c1-882e-c9127b336a3c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:12:41.706838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:12:41.719768 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Sync routers for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:12:41.732246 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:41.733301 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:41.793323 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] 10.222.0.82 "PUT /v2.0/routers/2fc03949-9293-45c1-882e-c9127b336a3c/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 3.0189509 May 25 16:12:41.795995 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 50998) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:41.939991 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:42.077801 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks [] for router, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:42.277900 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:42.297674 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36'}], 'enable_snat': True}, 'gw_port_id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:12:40Z', 'revision_number': 5, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port': {'id': '111d98cb-22ae-448d-b6d3-127948e3a74f', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:87:ae:60', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 363, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.36', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:06:46Z', 'updated_at': '2021-05-25T16:06:54Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:42.302114 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Checking router: 2fc03949-9293-45c1-882e-c9127b336a3c for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:12:42.311823 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Routers {'2fc03949-9293-45c1-882e-c9127b336a3c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:42.312625 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['2fc03949-9293-45c1-882e-c9127b336a3c'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:42.423590 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Deleting port 111d98cb-22ae-448d-b6d3-127948e3a74f {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:42.423973 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:42.473931 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] Set router 2fc03949-9293-45c1-882e-c9127b336a3c gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:42.585753 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:12:42Z', 'revision_number': 6, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:42.591771 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-87308c33-bdd4-4e0f-8268-194ceed2c502 None None] The sync data for ids ['2fc03949-9293-45c1-882e-c9127b336a3c'] in dv-cinder-786384-6-iscsi is: [{'id': '2fc03949-9293-45c1-882e-c9127b336a3c', 'name': 'tempest-TestShelveInstance-864022766-router', 'tenant_id': '649773106ce5438aaa40fb240586ad11', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:06:45Z', 'updated_at': '2021-05-25T16:12:42Z', 'revision_number': 6, 'project_id': '649773106ce5438aaa40fb240586ad11', 'gw_port_host': None}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:12:42.597939 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] For port 111d98cb-22ae-448d-b6d3-127948e3a74f, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=111d98cb-22ae-448d-b6d3-127948e3a74f,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:42.598522 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:42.756402 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Calling delete_port for 111d98cb-22ae-448d-b6d3-127948e3a74f owned by network:router_gateway {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:42.985428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:42.990127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-7e347444-4937-4968-8f17-8d9fe5707cf3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:43.041749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Pushing event deleted for resources: {'Port': ['ID=111d98cb-22ae-448d-b6d3-127948e3a74f,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:43.044762 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:43.151660 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.448765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.451803 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.571705 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.576342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Fanout notify agent at l3_agent the message router_deleted on router 2fc03949-9293-45c1-882e-c9127b336a3c {{(pid=99545) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:12:43.591507 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.593316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] 10.222.0.82 "DELETE /v2.0/routers/2fc03949-9293-45c1-882e-c9127b336a3c HTTP/1.1" status: 204 len: 168 time: 1.7963107 May 25 16:12:43.596428 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51006) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:43.667025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Deleting subnet 00016ef9-d427-4aab-906b-515d38fbeb52 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:12:43.671069 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.737807 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.810993 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d69a72d5-419a-489c-b21f-eee55561f332 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:43.833717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:43.877336 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: d69a72d5-419a-489c-b21f-eee55561f332 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:43.877797 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d69a72d5-419a-489c-b21f-eee55561f332 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:43.878529 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 111d98cb-22ae-448d-b6d3-127948e3a74f no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:43.937199 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 111d98cb-22ae-448d-b6d3-127948e3a74f {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:43.937733 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 111d98cb-22ae-448d-b6d3-127948e3a74f not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:44.188766 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.198471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:44.236589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=00016ef9-d427-4aab-906b-515d38fbeb52,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:44.238569 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.242856 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:44.246214 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-168eef79-d9a0-4d24-b8df-6fe5a8ebc0d9 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/00016ef9-d427-4aab-906b-515d38fbeb52 HTTP/1.1" status: 204 len: 168 time: 0.6480653 May 25 16:12:44.248988 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51020) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:44.251203 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-942a768e-d1eb-4093-92f7-2a8edac022eb None None] Network a333d7d4-c695-4061-8c6b-d08cd46bd9cf requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:12:44.373525 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.457450 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Deleting port ba27d822-ea42-4755-a4f4-b477a76c022b {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:44.458154 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.542596 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] DHCP port deletion for a333d7d4-c695-4061-8c6b-d08cd46bd9cf request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:12:44.570899 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] For port ba27d822-ea42-4755-a4f4-b477a76c022b, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=ba27d822-ea42-4755-a4f4-b477a76c022b,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:44.571251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.615586 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Deleting port ba27d822-ea42-4755-a4f4-b477a76c022b {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:44.615981 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.703419 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Calling delete_port for ba27d822-ea42-4755-a4f4-b477a76c022b owned by network:dhcp {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:44.727080 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] For port ba27d822-ea42-4755-a4f4-b477a76c022b, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=ba27d822-ea42-4755-a4f4-b477a76c022b,segment=NetworkSegment(03694269-0677-408f-bbe1-50be69716a73),segment_id=03694269-0677-408f-bbe1-50be69716a73)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:44.727487 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.809166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.813990 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:44.853905 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Calling delete_port for ba27d822-ea42-4755-a4f4-b477a76c022b owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:44.863059 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Pushing event deleted for resources: {'Port': ['ID=ba27d822-ea42-4755-a4f4-b477a76c022b,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:44.867183 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] No DVR routers for this DVR port ba27d822-ea42-4755-a4f4-b477a76c022b on host dv-cinder-786384-6-iscsi {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:44.898441 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.898811 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.899216 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.899573 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.899931 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.900291 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.900741 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.901037 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.905469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.905469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.905469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.905469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:12:44.908136 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:44.911394 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:44.917503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d1c60c31-3f17-42a1-8218-d09981326fe4 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:44.971264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Pushing event deleted for resources: {'Port': ['ID=ba27d822-ea42-4755-a4f4-b477a76c022b,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:44.973634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.056s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:44.974442 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] No DVR routers for this DVR port ba27d822-ea42-4755-a4f4-b477a76c022b on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:45.005108 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.018794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['a333d7d4-c695-4061-8c6b-d08cd46bd9cf']}} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:45.031301 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '03694269-0677-408f-bbe1-50be69716a73') {'for_net_delete': True} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:45.031591 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '03694269-0677-408f-bbe1-50be69716a73') {} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:45.043876 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3244641'] for segment, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.061968 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.070581 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Releasing vlan 906 on physical network physnet1 to pool {{(pid=99544) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:12:45.071097 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-3070910'] for segment, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: Traceback (most recent call last): May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: timer() May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: cb(*args, **kw) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: return func(*args, **kwargs) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: self._notify() May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: self.callback(batched_events) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: event.method(event) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: return f(self, *a, **k) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._get(url).json() May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: return self.request(url, 'GET', **kwargs) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:12:45.084870 dv-cinder-786384-6-iscsi neutron-server[99544]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:12:45.265810 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.270022 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-fa7409b3-56a9-47c8-bf09-950ebe40375f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:45.286278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.293584 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Pushing event deleted for resources: {'Network': ['ID=a333d7d4-c695-4061-8c6b-d08cd46bd9cf,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:45.298591 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:45.304679 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-47fb2cd7-7fb0-4b92-8415-7ffaf83e60a7 admin admin] 10.222.0.82 "DELETE /v2.0/networks/a333d7d4-c695-4061-8c6b-d08cd46bd9cf HTTP/1.1" status: 204 len: 168 time: 1.0545449 May 25 16:12:45.659100 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51036) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:45.710054 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9cbd166e-adf2-422c-b633-1dd941b0702e admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=5d4b9c0f27af44a9ace8862a2133a0cb&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0497153 May 25 16:12:45.713167 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51040) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:45.812887 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.817519 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device ba27d822-ea42-4755-a4f4-b477a76c022b no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:45.851455 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: ba27d822-ea42-4755-a4f4-b477a76c022b {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:45.851861 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device ba27d822-ea42-4755-a4f4-b477a76c022b not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:45.855760 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:45.988384 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51046) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:46.008438 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.018038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-463045c5-a64d-4485-ba1e-15e1da342b68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:46.042544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f1048f85-f0d1-40c6-9d99-c667080ff7b6,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:46.046592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.031s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:46.056637 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.061415 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-02803bb5-ab5e-4213-915d-d63c0d31453d admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f1048f85-f0d1-40c6-9d99-c667080ff7b6 HTTP/1.1" status: 204 len: 168 time: 0.3439853 May 25 16:12:46.117773 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.134166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:46.138969 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Usage tracker for resource:security_group_rule and tenant:e1ab49a6de8f4f66b2844438501769c6 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:46.141927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:46.152158 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:46.239855 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.331017 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.338976 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-21682c3d-3698-4366-b4c5-e7d7b0bbf877 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:46.385340 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=e1ab49a6de8f4f66b2844438501769c6&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3957238 May 25 16:12:46.387024 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=d28e05b4-54f1-41e7-8b31-5ebcceff43ea,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:46.389162 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51060) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:46.390392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:46.497195 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.527672 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.753667 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.758800 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-aea1e1ce-dfb6-4f1d-ab16-532e3f7523e6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:46.770593 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=d28e05b4-54f1-41e7-8b31-5ebcceff43ea,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:46.773579 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.015s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:46.775602 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:46.779422 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-f2c18433-2ab1-4e2a-bd95-7a62f4eb59d6 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/d28e05b4-54f1-41e7-8b31-5ebcceff43ea HTTP/1.1" status: 204 len: 168 time: 0.3870614 May 25 16:12:46.984978 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51070) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:47.036988 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-4232dd6a-079f-470b-887e-a455ccdabcea admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=649773106ce5438aaa40fb240586ad11&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0507529 May 25 16:12:47.039632 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51074) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:47.102744 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:47.141364 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:47.285910 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:47.291733 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9c825dba-13a0-47e4-8a40-59e2ddfaab78 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:47.304654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=627fbdb9-a018-40b0-8429-3985a3de0326,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:47.309026 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:47.313369 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:47.314947 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-b73faa0d-76e8-4047-9a73-17e6adcc3298 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/627fbdb9-a018-40b0-8429-3985a3de0326 HTTP/1.1" status: 204 len: 168 time: 0.2742383 May 25 16:12:48.270525 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:12:48.951427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51116) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:49.080131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.100251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:49.104326 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Usage tracker for resource:security_group_rule and tenant:2efaae06720c48ecb5c31da602bc6caa is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:49.107373 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:49.113587 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:49.213539 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.265185 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.274411 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e182fd26-a814-457a-89ff-422bd0fd8cf4 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:49.315658 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=2efaae06720c48ecb5c31da602bc6caa&name=default HTTP/1.1" status: 200 len: 2979 time: 0.3630652 May 25 16:12:49.319951 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51122) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:49.322082 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=977e8315-fff4-484c-81ee-e12a64731177,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:49.327243 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6f0d7d5b-b6ce-4813-97a4-73a0e54dd713 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:49.386224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.420722 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.574452 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.578398 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4af78b5f-6b74-4ece-b8b6-53964d43450a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:49.592838 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=977e8315-fff4-484c-81ee-e12a64731177,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:49.595971 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:49.598348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:49.599832 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8f0599d9-80ef-4369-bb7d-04e9ef22aa90 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/977e8315-fff4-484c-81ee-e12a64731177 HTTP/1.1" status: 204 len: 168 time: 0.2782531 May 25 16:12:54.938626 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1875e416-db31-479f-8710-79018c08b71e req-dab08903-bf3e-4635-b110-06a3d2a5365c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:54.940066 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1875e416-db31-479f-8710-79018c08b71e req-dab08903-bf3e-4635-b110-06a3d2a5365c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.0839705 May 25 16:12:54.987236 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1875e416-db31-479f-8710-79018c08b71e req-af5a896e-7350-49fe-93c4-23eb7194c00c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411658 May 25 16:12:56.193309 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fd64d88c-a896-4962-a4bc-944bacf42b51 req-aee25741-f411-4f80-9305-01a4b5fd2e32 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:56.196053 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd64d88c-a896-4962-a4bc-944bacf42b51 req-aee25741-f411-4f80-9305-01a4b5fd2e32 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.0891824 May 25 16:12:56.245603 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fd64d88c-a896-4962-a4bc-944bacf42b51 req-5cf5e6d6-d3da-456a-a0f9-73fd491b2a7d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0416319 May 25 16:12:56.532949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-6c60861b-d7e9-49b5-951b-f4c05a84b675 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:56.534397 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-6c60861b-d7e9-49b5-951b-f4c05a84b675 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.0811138 May 25 16:12:56.682888 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port 203fb615-7091-4405-8eb8-c2619684c5d1 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:12:56.683469 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:56.827816 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] For port 203fb615-7091-4405-8eb8-c2619684c5d1, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=203fb615-7091-4405-8eb8-c2619684c5d1,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:12:56.828198 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:57.265389 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:57.266095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:57.266095 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP 391d4545-674a-434a-9d8e-d2f524b79a84 disassociated. External IP: 172.27.2.79, port: 203fb615-7091-4405-8eb8-c2619684c5d1. May 25 16:12:57.266496 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for 203fb615-7091-4405-8eb8-c2619684c5d1 owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:12:57.457528 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e1a58fce-034f-4c41-ae81-fc3e9ccea97f req-8b9633c0-92f0-47eb-9f9e-e909717d8511 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:12:57.459065 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e1a58fce-034f-4c41-ae81-fc3e9ccea97f req-8b9633c0-92f0-47eb-9f9e-e909717d8511 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 945 time: 0.0961754 May 25 16:12:57.508030 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e1a58fce-034f-4c41-ae81-fc3e9ccea97f req-40850f67-d72e-4169-9bb4-2730df6fbbee tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0426793 May 25 16:12:57.599712 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:57.607189 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:57.654435 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=203fb615-7091-4405-8eb8-c2619684c5d1,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:57.658045 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:57.850714 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:12:58.045095 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:12:58.055664 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:12:58.066935 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:58.070969 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:58.138890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] No DVR routers for this DVR port 203fb615-7091-4405-8eb8-c2619684c5d1 on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:12:58.138890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:58.138890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'name': 'network-vif-deleted', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:12:58.160994 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-84f42e30-bea3-4212-8b60-f2ac3f589443 req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "DELETE /v2.0/ports/203fb615-7091-4405-8eb8-c2619684c5d1 HTTP/1.1" status: 204 len: 173 time: 1.6207209 May 25 16:12:58.200784 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '95d40cc6-3c36-4c64-9e63-7b41b72a0758', 'name': 'network-vif-deleted', 'tag': '203fb615-7091-4405-8eb8-c2619684c5d1', 'status': 'completed', 'code': 200} May 25 16:12:58.314431 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:58.657329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:58.657329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:12:58.665833 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:12:58.666573 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:58.683911 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-5832a2c5-8492-45bb-bb21-f06643a086ba req-98c90a99-0e92-462c-96f1-59d92f751c5f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:12:58.768099 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-5832a2c5-8492-45bb-bb21-f06643a086ba req-98c90a99-0e92-462c-96f1-59d92f751c5f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:12:58.769457 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-5832a2c5-8492-45bb-bb21-f06643a086ba req-98c90a99-0e92-462c-96f1-59d92f751c5f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=95d40cc6-3c36-4c64-9e63-7b41b72a0758 HTTP/1.1" status: 200 len: 210 time: 0.1366184 May 25 16:12:58.853506 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:12:59.149228 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51188) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:59.236264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:12:59.242194 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e54c1cf-8a72-4a65-b337-50f992157346 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:12:59.313834 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.333433 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:12:59.338289 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Usage tracker for resource:security_group_rule and tenant:c9cdd4be20374d84ac20ab6d50cfecad is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:12:59.342244 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:12:59.349093 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:12:59.456433 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.511935 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-8658ee66-519f-4e6d-b0fd-3eb8e8d1e4e3 None None] New status for floating IP 391d4545-674a-434a-9d8e-d2f524b79a84: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:12:59.582997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.592474 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d87f0c17-9a6a-4e46-a31a-88e1009ab867 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:59.634757 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=c9cdd4be20374d84ac20ab6d50cfecad&name=default HTTP/1.1" status: 200 len: 2979 time: 0.4844861 May 25 16:12:59.636541 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=2ad93c1e-8982-4688-8a99-2b54469e838d,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:59.638462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51194) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:12:59.641820 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-cb583115-58f1-41d2-afc6-76cad11bae40 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:59.704010 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.732098 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.824352 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 203fb615-7091-4405-8eb8-c2619684c5d1 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:12:59.868811 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 203fb615-7091-4405-8eb8-c2619684c5d1 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:12:59.869216 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 203fb615-7091-4405-8eb8-c2619684c5d1 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:12:59.880103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.889308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-729636aa-b0a2-49bd-bad5-fe287e8f8592 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:12:59.910544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=2ad93c1e-8982-4688-8a99-2b54469e838d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:12:59.915418 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.026s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:12:59.919619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:12:59.921200 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-515c7437-252e-4742-8612-8d95f019d38b admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/2ad93c1e-8982-4688-8a99-2b54469e838d HTTP/1.1" status: 204 len: 168 time: 0.2813294 May 25 16:13:00.173486 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-ad070c3a-4231-4d10-a7bb-9a067fb5a215 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-821231136&fields=id&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 267 time: 0.0408623 May 25 16:13:00.333591 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-80ac2348-b4b9-46ba-b8fb-91b1f91cb99d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:00.335116 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-80ac2348-b4b9-46ba-b8fb-91b1f91cb99d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.1550214 May 25 16:13:00.351711 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-b214829b-d8e5-4296-a922-4ece723bed8f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/quotas/a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 375 time: 0.0102375 May 25 16:13:00.450543 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-7128ffdc-ae29-4688-ac44-1fe7dee59f7b tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&fields=id HTTP/1.1" status: 200 len: 348 time: 0.0925467 May 25 16:13:00.710553 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-1e95298c-ad8c-4dd4-b302-6e164e32bcd6 service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2467341 May 25 16:13:00.863353 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0754fca5-b7a3-4356-bade-d18691ee7578 service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1459870 May 25 16:13:01.187259 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51232) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:01.307845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-1030053623-network', 'tenant_id': '7413f95c7d284674bb0d587de6b42810'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:01.312902 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:01.316685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Usage tracker for resource:network and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:01.319541 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:01.326552 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:01.428128 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Made reservation on behalf of 7413f95c7d284674bb0d587de6b42810 for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:01.430448 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.443711 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.459160 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:01.463263 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:01.466255 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:01.472698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:01.591440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-59d031f9-e2e7-4dc9-9c1f-767d88b46682 req-2054ba1f-9a24-4f5f-a8f5-ffefd486b943 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 210 time: 0.0614376 May 25 16:13:01.631657 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.696640 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.699838 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9be8dbb2-40a9-4013-9930-29769e4632c8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:01.727749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] vlan segment allocate from pool success with {'vlan_id': 916, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:13:01.737461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=8286425c-b378-48e3-9125-58c869823a8e,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:01.746865 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.747226 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Added segment 6075f059-4c81-4008-876b-49bbdcfe295e of type vlan for network ceb9e841-143d-468e-800d-756d578d900d May 25 16:13:01.749073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.749466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.749833 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.750195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.750546 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.750918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.751293 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.751629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.751973 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.752315 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.752658 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.753001 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:01.760502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.061s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:01.785932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.933984 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.935733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:01.936627 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:01.943359 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:02.046832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Pushing event updated for resources: {'Network': ['ID=ceb9e841-143d-468e-800d-756d578d900d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:02.049621 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:02.057467 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network ceb9e841-143d-468e-800d-756d578d900d details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:13:02.100262 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-960dfb18-b39d-4a69-94e5-03bcbcb151be admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 888 time: 0.9117537 May 25 16:13:02.103915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51248) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:02.111130 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Request body: {'subnet': {'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-1030053623-subnet', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:02.116555 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:02.121203 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Usage tracker for resource:subnet and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:02.125045 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:02.131972 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:02.242392 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Made reservation on behalf of 7413f95c7d284674bb0d587de6b42810 for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:02.244994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:02.783187 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-28df758f-62b6-440e-abf3-f4c40c181fec req-9e686706-b49f-4fd8-8ea7-8de208325b77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 210 time: 0.0622163 May 25 16:13:02.797533 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:02.801852 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fea74d1-949e-4c86-892c-ccbee1b57879 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:02.883060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Pushing event updated for resources: {'Subnet': ['ID=70734104-38bb-429b-abc3-63fbc11e4122,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:02.887574 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:02.894184 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Checking segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:02.894550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Network hostable DHCP agents. Network: ceb9e841-143d-468e-800d-756d578d900d, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:13:03.014257 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Network ceb9e841-143d-468e-800d-756d578d900d is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:13:03.156465 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, ceb9e841-143d-468e-800d-756d578d900d) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:03.168737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:03.168737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.170007 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-113e0181-fbce-42eb-9d58-d6c0eab392c2 None None] Network ceb9e841-143d-468e-800d-756d578d900d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:13:03.274545 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2d2bcf8a-f3bd-486b-83d5-f31d7f7f755f admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 851 time: 1.1687775 May 25 16:13:03.277689 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51254) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:03.283694 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:03.289580 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:03.295265 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Usage tracker for resource:router and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:03.299002 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:03.306274 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:03.423072 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.424167 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.444604 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.448988 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-ceb9e841-143d-468e-800d-756d578d900d', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:13:03.449812 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.674158 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Allocated IP 172.20.0.2 (ceb9e841-143d-468e-800d-756d578d900d/70734104-38bb-429b-abc3-63fbc11e4122/b8e1535c-25c9-469e-8c50-6b96e5f67f74) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:03.919775 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:03.935699 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Transition to ACTIVE for port object b8e1535c-25c9-469e-8c50-6b96e5f67f74 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:03.960892 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Transition to ACTIVE for port object b8e1535c-25c9-469e-8c50-6b96e5f67f74 will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:04.018266 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.063014 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.067019 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:04.107233 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-77dbf7aa-569a-46f6-8085-f22bab4f0b7f req-f21afcc5-9ca6-4827-a3e9-b837e24444a2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 210 time: 0.2085781 May 25 16:13:04.148714 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Network ceb9e841-143d-468e-800d-756d578d900d is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:04.149505 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Pushing event updated for resources: {'Port': ['ID=b8e1535c-25c9-469e-8c50-6b96e5f67f74,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:04.153066 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:04.155672 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:04.162841 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.096s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:04.252791 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Attempting to bind port b8e1535c-25c9-469e-8c50-6b96e5f67f74 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:04.253663 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Attempting to bind port b8e1535c-25c9-469e-8c50-6b96e5f67f74 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:04.253663 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Attempting to bind port b8e1535c-25c9-469e-8c50-6b96e5f67f74 on network ceb9e841-143d-468e-800d-756d578d900d {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:04.261333 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 12, 38), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:04.261719 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Checking segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:04.262146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.262551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Bound using segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:04.263091 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Bound port: b8e1535c-25c9-469e-8c50-6b96e5f67f74, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:04.263486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.362063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] For port b8e1535c-25c9-469e-8c50-6b96e5f67f74, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:04.365746 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] For port b8e1535c-25c9-469e-8c50-6b96e5f67f74, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b8e1535c-25c9-469e-8c50-6b96e5f67f74,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:04.430725 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.451324 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-39ddd376-16a7-4771-a70a-b1373e5b4436 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:04.453547 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-39ddd376-16a7-4771-a70a-b1373e5b4436 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.1650162 May 25 16:13:04.463262 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.468074 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.468888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-dc5f5c33-9c85-4d5f-a72b-fd06abe534fe None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:04.516379 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-c1e578f5-b987-45c2-8dfa-208dbee787fd tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 379 time: 0.0476201 May 25 16:13:04.525539 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'port': {'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'admin_state_up': True, 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'security_groups': ['dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64']}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:04.549548 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Pushing event updated for resources: {'Port': ['ID=b8e1535c-25c9-469e-8c50-6b96e5f67f74,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:04.555268 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.086s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:04.640804 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Allocated IP 172.27.2.138 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/199437ae-99fb-4810-a6e7-6003a7272edd) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:04.701061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:04.712950 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource port. Total usage: 3; quota limit: 500; headroom:497 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:04.788361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Made reservation on behalf of a1e31a264b7949bea6aa47cb7781106b for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:04.795071 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.867857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.981816 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:04.992245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-902e5010-cfbd-45b4-b720-beb44f23c6c4 tempest-VolumesSnapshotTestJSON-85714828 tempest-VolumesSnapshotTestJSON-85714828-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:05.065129 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:05.068719 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:05.073022 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:05.105085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Pushing event updated for resources: {'Port': ['ID=199437ae-99fb-4810-a6e7-6003a7272edd,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:05.111801 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.123s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:05.149622 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.20.0.5 (6c61cff6-814b-4af0-8599-e026b705cc35/a4d8980e-d38c-4165-af50-ed2f39dc708c/4b64460a-d17d-4ac3-82d0-d9990d9be8b4) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:05.567325 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b7bf2752-955d-4591-98a4-418c1f694fc3 req-264cd604-a4be-4f6a-985c-7006ece966a5 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 210 time: 0.1303287 May 25 16:13:05.577317 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:05.598206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Transition to ACTIVE for port object 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:05.681291 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-72bdd8ac-8f36-418c-b380-8c3e86fb4f8d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:05.697891 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-d2ef37c6-da7c-4937-90ee-e4a6ddaf1293 None None] Network ceb9e841-143d-468e-800d-756d578d900d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:13:05.755581 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:05.765044 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8f26a36e-819b-4708-8fc0-567090b25939 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:05.897089 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:05.901618 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:05.906217 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:05.910216 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b8e1535c-25c9-469e-8c50-6b96e5f67f74 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:05.931570 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:05.969150 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:05.970979 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-fa787d1a-b787-4c83-824e-7aefc6d4030c None None] Provisioning for port b8e1535c-25c9-469e-8c50-6b96e5f67f74 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:05.979806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.213s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:06.019132 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:06.020261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.151766 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-d2231305-5865-4d9a-b605-801557ba3bef tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.6300108 May 25 16:13:06.153415 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port b8e1535c-25c9-469e-8c50-6b96e5f67f74 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:06.159444 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port b8e1535c-25c9-469e-8c50-6b96e5f67f74 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:06.159898 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.171655 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Request body: {'port': {'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:06.339086 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.352926 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.413083 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.436241 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port b8e1535c-25c9-469e-8c50-6b96e5f67f74, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b8e1535c-25c9-469e-8c50-6b96e5f67f74,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:06.485903 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-1c8cd800-9609-45f9-a069-323811879e5c None None] Provisioning for port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:06.491723 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-1c8cd800-9609-45f9-a069-323811879e5c None None] Provisioning complete for port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:06.492000 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-1c8cd800-9609-45f9-a069-323811879e5c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.526397 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.540325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-92c93c12-5595-45ad-8343-a19734312b64 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:06.614782 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-1c8cd800-9609-45f9-a069-323811879e5c None None] Port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:06.629040 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=b8e1535c-25c9-469e-8c50-6b96e5f67f74,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:06.632984 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.093s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:06.739119 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-a4112a84-fa15-4d44-ad1f-36c3800b0ddc None None] Provisioning complete for port b8e1535c-25c9-469e-8c50-6b96e5f67f74 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:06.740186 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a4112a84-fa15-4d44-ad1f-36c3800b0ddc None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.800046 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Router 0492512e-ccab-4c81-9788-1e5b615f9dc3 is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99544) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:13:06.853151 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:13:06.862663 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.887766 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Transition to ACTIVE for port object 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:06.902138 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9be55ae2-f8d6-415c-bfbf-7bf12b589332 req-8f30d10d-ade7-49cc-a89c-e5c9e7c32b98 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:06.902757 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-9be55ae2-f8d6-415c-bfbf-7bf12b589332 req-8f30d10d-ade7-49cc-a89c-e5c9e7c32b98 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:13:06.988446 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:06.994478 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-dbe99f51-6e9d-4d28-a7ed-4710d3362abd service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:07.013117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:13:07.023649 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Sync routers for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:13:07.034024 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:07.034715 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 4, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 4, "log_agent_heartbeats": false, "routers": 4},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:12:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:07.049902 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-9be55ae2-f8d6-415c-bfbf-7bf12b589332 req-8f30d10d-ade7-49cc-a89c-e5c9e7c32b98 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:13:07.051107 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9be55ae2-f8d6-415c-bfbf-7bf12b589332 req-8f30d10d-ade7-49cc-a89c-e5c9e7c32b98 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 931 time: 0.3219421 May 25 16:13:07.123121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:07.125172 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d789ca1d-a1d0-4ff3-b305-71fff936dc87 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 927 time: 3.8462460 May 25 16:13:07.127941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51268) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:07.155590 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Pushing event updated for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:07.167701 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:07.263006 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9be55ae2-f8d6-415c-bfbf-7bf12b589332 req-48e3c104-4055-4d09-b2d3-2388139467ee tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2010083 May 25 16:13:07.329510 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Attempting to bind port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:07.329510 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Attempting to bind port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:07.329510 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Attempting to bind port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:07.334689 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 on network 6c61cff6-814b-4af0-8599-e026b705cc35 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:13:07.335046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Attempting to bind port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:07.340771 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 12, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:07.341103 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:07.341488 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Bound using segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:07.341917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Bound port: 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:07.342265 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:07.369359 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:07.440766 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:07.443934 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:07.533984 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:07.539578 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c086b0ba-5a2c-478f-b93b-da09625d289a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:07.600301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:07.600871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:07.602267 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] 10.222.0.42 "PUT /v2.0/ports/4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 1093 time: 1.4377854 May 25 16:13:07.602781 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:13:07.629193 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Pushing event updated for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:07.641468 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.102s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:07.694360 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'status': 'completed', 'code': 200} May 25 16:13:07.889075 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-0c55e12d-45ac-4a8f-83ba-c16033a42b39 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:07.890983 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b8e1535c-25c9-469e-8c50-6b96e5f67f74 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:07.891828 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-0c55e12d-45ac-4a8f-83ba-c16033a42b39 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1096 time: 0.1528964 May 25 16:13:08.007390 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:05Z', 'revision_number': 3, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:04Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:08.011349 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port b8e1535c-25c9-469e-8c50-6b96e5f67f74 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:08.011868 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.043250 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Checking router: 0492512e-ccab-4c81-9788-1e5b615f9dc3 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:13:08.043634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Updating router 0492512e-ccab-4c81-9788-1e5b615f9dc3 port 199437ae-99fb-4810-a6e7-6003a7272edd binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:13:08.100550 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-97faac85-90ef-49f1-a0cf-e03a613cfe37 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.102317 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-97faac85-90ef-49f1-a0cf-e03a613cfe37 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=a1e31a264b7949bea6aa47cb7781106b&shared=False HTTP/1.1" status: 200 len: 912 time: 0.1973102 May 25 16:13:08.139626 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.194862 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.279441 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-1c760a9d-ce4e-4bc1-8183-9376ccb978f4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.281571 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a5f5095-9e6b-4b80-9564-4dcd33297585 req-1c760a9d-ce4e-4bc1-8183-9376ccb978f4 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.1725972 May 25 16:13:08.357740 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.462694 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-af5840bb-e961-4e30-b7b2-6bf6fe10f29c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.527862 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-af246f95-cde8-4ff1-80ed-a346eaf5136f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.529479 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-af246f95-cde8-4ff1-80ed-a346eaf5136f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1096 time: 0.1689422 May 25 16:13:08.645328 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.668094 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Transition to ACTIVE for port object 199437ae-99fb-4810-a6e7-6003a7272edd will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:08.706812 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Allocated IP 172.20.0.1 (ceb9e841-143d-468e-800d-756d578d900d/70734104-38bb-429b-abc3-63fbc11e4122/61c3265e-fd7c-491c-8373-f3de5eca4718) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:08.733191 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-bab74610-f185-4db7-a550-b0c463880a19 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 217 time: 0.1900880 May 25 16:13:08.750186 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3ed1ba79-a2c2-46b7-b174-326cc8d6935a req-c5d1f4af-5c25-4faf-be98-486ce77b5636 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.750186 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ed1ba79-a2c2-46b7-b174-326cc8d6935a req-c5d1f4af-5c25-4faf-be98-486ce77b5636 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.3023593 May 25 16:13:08.756880 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.765614 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:08.855798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-06ba56fd-cb10-4ac2-bf1b-8b83d36024c6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.857328 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-06ba56fd-cb10-4ac2-bf1b-8b83d36024c6 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.1034706 May 25 16:13:08.860705 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3ed1ba79-a2c2-46b7-b174-326cc8d6935a req-26838b7e-20fa-472f-80aa-3e16bee782e7 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0715301 May 25 16:13:08.879792 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Attempting to bind port 199437ae-99fb-4810-a6e7-6003a7272edd on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:08.880170 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Attempting to bind port 199437ae-99fb-4810-a6e7-6003a7272edd by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:08.880621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Attempting to bind port 199437ae-99fb-4810-a6e7-6003a7272edd on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:08.897108 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:08.897816 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:08.898613 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.899035 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:08.899855 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Bound port: 199437ae-99fb-4810-a6e7-6003a7272edd, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:08.900256 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.907459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:08.927375 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.provisioning_blocks [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Transition to ACTIVE for port object 61c3265e-fd7c-491c-8373-f3de5eca4718 will not be triggered until provisioned by entity DHCP. {{(pid=99544) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:08.952742 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-39178e57-45e3-4d87-8dfa-3cc424dca035 None None] Pushing event updated for resources: {'Port': ['ID=199437ae-99fb-4810-a6e7-6003a7272edd,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:08.957949 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-39178e57-45e3-4d87-8dfa-3cc424dca035 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.193s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:08.966187 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-4cebe986-9d48-4219-b09f-02c5f1db31ef service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:08.968524 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-4cebe986-9d48-4219-b09f-02c5f1db31ef service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1036048 May 25 16:13:09.008502 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] For port 199437ae-99fb-4810-a6e7-6003a7272edd, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:09.012582 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] For port 199437ae-99fb-4810-a6e7-6003a7272edd, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=199437ae-99fb-4810-a6e7-6003a7272edd,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:09.031806 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.037475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:09.091796 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.097700 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-39178e57-45e3-4d87-8dfa-3cc424dca035 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:09.111656 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Network ceb9e841-143d-468e-800d-756d578d900d is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:09.115431 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:09.120921 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:09.131412 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-f8d51c31-d364-4833-afb0-6d0f269115ca service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1566339 May 25 16:13:09.145913 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.165526 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Pushing event updated for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:09.177052 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:09.186127 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:09.186761 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:09.195724 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.299230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-39178e57-45e3-4d87-8dfa-3cc424dca035 None None] Pushing event updated for resources: {'Port': ['ID=199437ae-99fb-4810-a6e7-6003a7272edd,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:09.313454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-39178e57-45e3-4d87-8dfa-3cc424dca035 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.216s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:09.335055 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-200c8d66-8fa8-4f40-9562-31f3f093f53a req-73fa266b-ee75-43f5-8113-d3c9d0ddd920 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1970441 May 25 16:13:09.437185 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:09.542219 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-29880739-d134-445a-bc5d-65236bef43d1 None None] Provisioning for port 61c3265e-fd7c-491c-8373-f3de5eca4718 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:09.573325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-29880739-d134-445a-bc5d-65236bef43d1 None None] Provisioning complete for port 61c3265e-fd7c-491c-8373-f3de5eca4718 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:09.573844 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-29880739-d134-445a-bc5d-65236bef43d1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.648654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.750746 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-29880739-d134-445a-bc5d-65236bef43d1 None None] Port 61c3265e-fd7c-491c-8373-f3de5eca4718 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:09.852728 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:05Z', 'revision_number': 3, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:09.858255 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-29f22082-edfc-4086-a150-b5e49c1e7399 None None] The sync data for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi is: [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:05Z', 'revision_number': 3, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:13:09.926536 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:09.996965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:10.025045 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6cd8a630-7e78-46dc-bab0-d9dae70f1299 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:10.142541 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Pushing event updated for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:10.143052 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dac5b21a-d94d-4612-8eae-f534dfa376bc req-96b5e569-7370-4bc3-a426-ceccc4e221b5 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:10.148316 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dac5b21a-d94d-4612-8eae-f534dfa376bc req-96b5e569-7370-4bc3-a426-ceccc4e221b5 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.1423135 May 25 16:13:10.154119 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:10.161395 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.136s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:10.236267 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dac5b21a-d94d-4612-8eae-f534dfa376bc req-cd73ff6e-08dd-48c2-b776-9618cf3a6856 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0709260 May 25 16:13:10.501799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Network ceb9e841-143d-468e-800d-756d578d900d is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:10.503108 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:10.583272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:10.715675 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Router 0492512e-ccab-4c81-9788-1e5b615f9dc3 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:13:10.793534 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-047b76d6-c745-4213-8565-e53dd28d36a3 None None] Provisioning complete for port 61c3265e-fd7c-491c-8373-f3de5eca4718 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:10.794282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-047b76d6-c745-4213-8565-e53dd28d36a3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:10.873539 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-047b76d6-c745-4213-8565-e53dd28d36a3 None None] Port 61c3265e-fd7c-491c-8373-f3de5eca4718 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:10.917525 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:13:10.926706 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-598e3cbf-645e-4432-8203-48d064d81eae admin admin] 10.222.0.82 "PUT /v2.0/routers/0492512e-ccab-4c81-9788-1e5b615f9dc3/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.7977209 May 25 16:13:11.125149 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:11.498502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9d637265-62d9-4cc9-a5cc-357bb6434808 req-cd7327ae-963a-4500-8eb3-6e0692d02464 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:11.500729 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9d637265-62d9-4cc9-a5cc-357bb6434808 req-cd7327ae-963a-4500-8eb3-6e0692d02464 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.1080389 May 25 16:13:11.562225 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9d637265-62d9-4cc9-a5cc-357bb6434808 req-cacf0fbc-1ccc-4c49-b319-123687d18e0c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0497148 May 25 16:13:11.599150 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Sync routers for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:13:11.609206 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:11.609834 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:11.752538 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51304) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:11.758535 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Request body: {'network': {'name': 'tempest-TestMinimumBasicScenario-299703303-network', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:11.764175 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:11.770689 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Usage tracker for resource:network and tenant:92165e91edbb47c88928d15e0da2d7bd is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:11.777080 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Quota usage for network was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:11.786289 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Attempting to reserve 1 items for resource network. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:11.807924 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:11.852063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 199437ae-99fb-4810-a6e7-6003a7272edd up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:11.939040 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Made reservation on behalf of 92165e91edbb47c88928d15e0da2d7bd for: {'network': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:11.941551 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_net-16139546'] for network, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:11.955799 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks [] for security_group, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:11.973030 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:11.977115 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Usage tracker for resource:security_group_rule and tenant:92165e91edbb47c88928d15e0da2d7bd is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:11.980822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Quota usage for security_group_rule was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:11.988115 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Attempting to reserve 4 items for resource security_group_rule. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:12.084272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks [] for security_group, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.096490 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 199437ae-99fb-4810-a6e7-6003a7272edd completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:12.101953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 199437ae-99fb-4810-a6e7-6003a7272edd triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:12.107167 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.143194 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 2, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:12.152001 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Checking router: 0492512e-ccab-4c81-9788-1e5b615f9dc3 for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:13:12.152001 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Updating router 0492512e-ccab-4c81-9788-1e5b615f9dc3 port 61c3265e-fd7c-491c-8373-f3de5eca4718 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:13:12.183234 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.186431 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f4061bad-255b-4751-afb4-d21d44ad16f8 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:12.219780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.helpers [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] vlan segment allocate from pool success with {'vlan_id': 933, 'physical_network': 'physnet1'} {{(pid=99545) allocate_partially_specified_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/helpers.py:153}} May 25 16:13:12.230483 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Pushing event updated for resources: {'SecurityGroup': ['ID=6b980970-f336-4735-a014-d970af478641,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:12.233015 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.233918 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:12.239904 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.240320 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.segments_db [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Added segment 22130cb2-3662-4892-b32e-70b172a78d26 of type vlan for network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 May 25 16:13:12.255052 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 199437ae-99fb-4810-a6e7-6003a7272edd, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=199437ae-99fb-4810-a6e7-6003a7272edd,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:12.261280 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.276960 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.services.auto_allocate.db._ensure_external_network_default_value_callback-8747111503153'] for network, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.336614 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.341852 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:12.396833 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238'] for network, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.397869 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:12.398645 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.405094 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0b0c6238-1336-46aa-b797-4d89cc04669a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:12.450257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=199437ae-99fb-4810-a6e7-6003a7272edd,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:12.456225 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.114s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:12.512688 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Pushing event updated for resources: {'Network': ['ID=4ae9aef2-8f6b-4c56-b394-f86738d1b047,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:12.515561 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.111s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:12.525664 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 details requested by agent ovs-agent-dv-cinder-786384-6-iscsi with host dv-cinder-786384-6-iscsi {{(pid=99546) get_network_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:232}} May 25 16:13:12.649024 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a5c7e759-1456-4bf8-9669-c1eaec2fe088 admin admin] 10.222.0.82 "POST /v2.0/networks HTTP/1.1" status: 201 len: 887 time: 0.8951306 May 25 16:13:12.652495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51312) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:12.659198 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Request body: {'subnet': {'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'cidr': '172.20.0.0/28', 'name': 'tempest-TestMinimumBasicScenario-299703303-subnet', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'ip_version': 4}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:12.665061 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:12.669898 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Usage tracker for resource:subnet and tenant:92165e91edbb47c88928d15e0da2d7bd is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:12.673734 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Quota usage for subnet was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:12.683980 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Attempting to reserve 1 items for resource subnet. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:12.726950 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.740963 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Transition to ACTIVE for port object 61c3265e-fd7c-491c-8373-f3de5eca4718 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:12.818049 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8dabd1a2-93fd-4143-91db-a362860f48a3 req-58bfbf44-0a63-470f-aa76-e4892ce7de3a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:12.819586 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8dabd1a2-93fd-4143-91db-a362860f48a3 req-58bfbf44-0a63-470f-aa76-e4892ce7de3a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.0951235 May 25 16:13:12.822375 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Made reservation on behalf of 92165e91edbb47c88928d15e0da2d7bd for: {'subnet': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:12.825780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Notify callbacks ['neutron.services.segments.plugin.SegmentHostRoutes.host_routes_before_create-2319288'] for subnet, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.903576 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:12.907928 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:12.985552 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8dabd1a2-93fd-4143-91db-a362860f48a3 req-d4de4124-383f-4b9c-89a4-0d83a260d4c3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1442301 May 25 16:13:13.109314 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Pushing event updated for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:13.127314 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.127654 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.127961 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.128288 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.128609 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.128931 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.129248 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.129564 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.129903 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.130236 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.130548 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.130858 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99547) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:13.170934 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.263s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:13.216615 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Attempting to bind port 61c3265e-fd7c-491c-8373-f3de5eca4718 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:13.217232 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Attempting to bind port 61c3265e-fd7c-491c-8373-f3de5eca4718 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:13.217817 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Attempting to bind port 61c3265e-fd7c-491c-8373-f3de5eca4718 on network ceb9e841-143d-468e-800d-756d578d900d {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:13.227543 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:13.228127 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Checking segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:13.228773 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:13.229403 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Bound using segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:13.230175 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Bound port: 61c3265e-fd7c-491c-8373-f3de5eca4718, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:13.230758 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:13.380560 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] For port 61c3265e-fd7c-491c-8373-f3de5eca4718, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:13.386500 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] For port 61c3265e-fd7c-491c-8373-f3de5eca4718, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=61c3265e-fd7c-491c-8373-f3de5eca4718,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:13.454207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_created-1892956', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_create-2319278'] for subnet, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:13.458736 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-bfa2c7d8-dbef-40b2-8508-82fcec969fae admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:13.465551 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:13.470798 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:13.544404 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Pushing event updated for resources: {'Subnet': ['ID=d4ddf00b-460c-41c8-8051-1d52a3935d5b,revision_number=0']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:13.546376 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:13.547053 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:13.548205 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:13.554758 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Checking segment: {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:13.555139 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Network hostable DHCP agents. Network: 4ae9aef2-8f6b-4c56-b394-f86738d1b047, hostable agents: ['cfa38b0b-c050-40ee-92d3-cce6adca5eb4'], hosted agents: [] {{(pid=99545) _get_network_hostable_dhcp_agents /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:329}} May 25 16:13:13.606176 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Pushing event updated for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:13.626058 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-1c0b530c-0528-480d-92ac-d06d87a6c733 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.155s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:13.651192 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 is scheduled to be hosted by DHCP agent cfa38b0b-c050-40ee-92d3-cce6adca5eb4 with binding_index 1 {{(pid=99545) bind /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:227}} May 25 16:13:13.736454 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Resources bound (agent type, host, resource id): (DHCP agent, dv-cinder-786384-6-iscsi, 4ae9aef2-8f6b-4c56-b394-f86738d1b047) {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:13.748940 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:13.750096 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:13.752634 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e39997aa-d516-4cdc-8061-b90f31c5185e None None] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:13:13.761744 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:13.840428 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 199437ae-99fb-4810-a6e7-6003a7272edd up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:13.945474 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-8613cf3f-7eee-43ea-b572-3c38714d1a41 admin admin] 10.222.0.82 "POST /v2.0/subnets HTTP/1.1" status: 201 len: 850 time: 1.2891867 May 25 16:13:13.945750 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51316) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:13.951301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Request body: {'router': {'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:13.956282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:13.960754 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Usage tracker for resource:router and tenant:92165e91edbb47c88928d15e0da2d7bd is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:13.964818 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Quota usage for router was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:13.971607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Attempting to reserve 1 items for resource router. Total usage: 0; quota limit: 100; headroom:100 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:13.998748 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 199437ae-99fb-4810-a6e7-6003a7272edd triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:13.999403 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.082592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._before_router_create-489025', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._check_router_request-2417'] for router, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.083944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._precommit_router_create-488001', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._set_distributed_flag-490654', 'neutron.services.l3_router.l3_router_plugin.RouterAvailabilityZoneMixin._process_az_request-16306803', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._set_router_provider-2410'] for router, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.115638 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Create dhcp port {'port': {'name': '', 'admin_state_up': True, 'device_id': 'dhcpcfa30929-4b91-5450-bc1d-643b73241a9c-4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b'}]}} from dv-cinder-786384-6-iscsi. {{(pid=99546) create_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:285}} May 25 16:13:14.116608 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.118685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks [] for router_controller, precommit_add_association {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.139087 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:13Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:14.144639 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-098393cb-40c5-4153-9251-dd55cc73f496 None None] The sync data for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi is: [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'DOWN', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:13Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:13:14.276857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c5580494-75d8-44b2-97fb-42de0682fc45 req-5ba9937e-47e1-4ed2-bb76-e17f51d66836 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:14.278493 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5580494-75d8-44b2-97fb-42de0682fc45 req-5ba9937e-47e1-4ed2-bb76-e17f51d66836 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.1388776 May 25 16:13:14.445905 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.db_base_plugin_common [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Allocated IP 172.20.0.2 (4ae9aef2-8f6b-4c56-b394-f86738d1b047/d4ddf00b-460c-41c8-8051-1d52a3935d5b/05f35bc2-759b-4098-ab8d-b513e2a36c4d) {{(pid=99546) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:14.694910 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks [] for port, precommit_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.699614 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c5580494-75d8-44b2-97fb-42de0682fc45 req-2e00efbb-50ce-438e-b8f6-2e195e8a4358 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.2453828 May 25 16:13:14.708894 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Transition to ACTIVE for port object 05f35bc2-759b-4098-ab8d-b513e2a36c4d will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:14.719050 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._handle_distributed_migration-490683', 'neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._validate_migration-489075', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._update_router_provider-2388'] for router, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.730319 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Transition to ACTIVE for port object 05f35bc2-759b-4098-ab8d-b513e2a36c4d will not be triggered until provisioned by entity DHCP. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:14.871860 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903'] for port, after_create {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:14.879143 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:14.976573 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 is already hosted by enough agents. {{(pid=99546) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:14.981741 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.scheduler.base_scheduler [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Resources bound (agent type, host, resource id): {{(pid=99546) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:14.986138 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Received port after_create {{(pid=99546) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:15.023845 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Pushing event updated for resources: {'Port': ['ID=05f35bc2-759b-4098-ab8d-b513e2a36c4d,revision_number=1']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:15.031391 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:15.145332 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Attempting to bind port 05f35bc2-759b-4098-ab8d-b513e2a36c4d on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:15.145719 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Attempting to bind port 05f35bc2-759b-4098-ab8d-b513e2a36c4d by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:15.146099 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Attempting to bind port 05f35bc2-759b-4098-ab8d-b513e2a36c4d on network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:15.153333 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:15.153866 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Checking segment: {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:15.154358 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.154909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Bound using segment: {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:15.155470 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Bound port: 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:15.155981 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.217950 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks [] for router_gateway, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.254842 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.259040 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] For port 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:15.263120 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] For port 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=05f35bc2-759b-4098-ab8d-b513e2a36c4d,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:15.385919 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.391198 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:15.438456 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Allocated IP 172.27.2.164 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:15.491857 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Pushing event updated for resources: {'Port': ['ID=05f35bc2-759b-4098-ab8d-b513e2a36c4d,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:15.504014 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:15.640195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.770363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:15.775340 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-04c96dbb-81f7-4ae9-8a33-cb19378f1a68 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:15.935490 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:15.936957 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 61c3265e-fd7c-491c-8373-f3de5eca4718 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:15.940295 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:15.943915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Pushing event updated for resources: {'Port': ['ID=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:15.946964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:15.976589 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.201s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:16.028930 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-37c56c75-81e1-4c86-91b1-f72a0c9c8a1d req-5cc71b9c-18f1-401f-aa08-27a57b964a72 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:16.030471 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-37c56c75-81e1-4c86-91b1-f72a0c9c8a1d req-5cc71b9c-18f1-401f-aa08-27a57b964a72 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.1862738 May 25 16:13:16.130553 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 61c3265e-fd7c-491c-8373-f3de5eca4718 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:16.137418 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 61c3265e-fd7c-491c-8373-f3de5eca4718 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:16.138075 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:16.305820 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:16.346572 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port 61c3265e-fd7c-491c-8373-f3de5eca4718, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=61c3265e-fd7c-491c-8373-f3de5eca4718,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:16.411169 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-37c56c75-81e1-4c86-91b1-f72a0c9c8a1d req-feb0fcf4-5615-43f5-bc57-968d29d98164 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.3273954 May 25 16:13:16.435077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:16.435077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 05f35bc2-759b-4098-ab8d-b513e2a36c4d up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:16.460077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:16.535158 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-491ef993-49d6-44d3-a017-b15be95b9be1 None None] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:13:16.655083 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:16.658325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.198s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:16.758145 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks [] for router_gateway, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:16.788785 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Provisioning for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:16.803566 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:16.809360 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Provisioning complete for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:16.814320 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:16.858128 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:16.858510 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.067833 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.091728 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.128835 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] For port 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=05f35bc2-759b-4098-ab8d-b513e2a36c4d,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:17.131805 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._after_router_create-489032', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_snat_interfaces_after_change-490744'] for router, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.219282 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.222625 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d59a5662-1b75-4cc1-944d-9c00804ba3f4 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:17.344045 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.db.api [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Retry wrapper got retriable exception: UPDATE statement on table 'standardattributes' expected to update 1 row(s); 0 were matched. {{(pid=99546) wrapped /usr/local/lib/python3.8/dist-packages/neutron_lib/db/api.py:183}} May 25 16:13:17.344045 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_db.api [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Performing DB retry for function neutron.plugins.ml2.plugin.Ml2Plugin.update_port_statuses {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} May 25 16:13:17.348659 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Pushing event updated for resources: {'Port': ['ID=05f35bc2-759b-4098-ab8d-b513e2a36c4d,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:17.351710 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-3ce70f75-e11c-456b-9e51-186cb5968c60 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.129s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:17.357736 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-191d19e1-c266-4d02-8031-8b386f5cb9fd None None] Provisioning complete for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:17.358069 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-191d19e1-c266-4d02-8031-8b386f5cb9fd None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.462689 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Router acd58228-b46f-4a8e-be41-0910702cff46 is scheduled to L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 with binding_index 1 {{(pid=99545) bind_router /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:210}} May 25 16:13:17.703537 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:13:17.715955 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Sync routers for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:13:17.726346 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:17.727017 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:17.733660 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-92634e72-18db-41f0-9540-13fab2768ec1 req-30ff52c9-3623-467e-a0c8-869cec854938 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:17.735345 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-92634e72-18db-41f0-9540-13fab2768ec1 req-30ff52c9-3623-467e-a0c8-869cec854938 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 943 time: 0.2108691 May 25 16:13:17.791035 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-92634e72-18db-41f0-9540-13fab2768ec1 req-08bfb37f-f516-4bd3-b65f-b486200756da tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0485377 May 25 16:13:17.833892 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:17.835132 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] 10.222.0.82 "POST /v2.0/routers HTTP/1.1" status: 201 len: 926 time: 3.8882310 May 25 16:13:17.845537 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51326) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:17.969888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:13:18.043113 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:18.197035 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 61c3265e-fd7c-491c-8373-f3de5eca4718 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:18.441384 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:18.481657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 61c3265e-fd7c-491c-8373-f3de5eca4718 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:18.482045 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:18.534117 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 902, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.5'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:13:18.632830 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:18.697972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:13:18.722967 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 3, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:15Z', 'revision_number': 1, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:18.731858 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 05f35bc2-759b-4098-ab8d-b513e2a36c4d up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:18.746010 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:18.763221 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:18.764267 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Checking router: acd58228-b46f-4a8e-be41-0910702cff46 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:13:18.764599 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Updating router acd58228-b46f-4a8e-be41-0910702cff46 port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 binding host dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:13:18.829157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:18.834906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:18.945241 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.040446 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Allocated IP 172.20.0.1 (4ae9aef2-8f6b-4c56-b394-f86738d1b047/d4ddf00b-460c-41c8-8051-1d52a3935d5b/bef09144-4688-468a-b1ed-5f1bf3929ba3) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:19.050899 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:19.059398 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.225s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:19.063915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port 05f35bc2-759b-4098-ab8d-b513e2a36c4d triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:19.063915 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.156054 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d99b3c1d-328e-405e-8098-7c72a1cabde9 req-e0241d3f-f6ce-4d2e-8634-710c2e230dea tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:19.157606 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d99b3c1d-328e-405e-8098-7c72a1cabde9 req-e0241d3f-f6ce-4d2e-8634-710c2e230dea tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 944 time: 0.2380013 May 25 16:13:19.168492 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['4b64460a-d17d-4ac3-82d0-d9990d9be8b4'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:13:19.258673 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d99b3c1d-328e-405e-8098-7c72a1cabde9 req-f7499fff-6e29-45af-9c43-3011e7e00067 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0915256 May 25 16:13:19.283221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.312477 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Transition to ACTIVE for port object bef09144-4688-468a-b1ed-5f1bf3929ba3 will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:19.396765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.401931 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:19.462073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:19.463379 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:19.465711 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:19.532992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Pushing event updated for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:19.542101 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.140s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:19.545261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._add_csnat_on_interface_create-491401'] for router_interface, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.658677 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks [] for port, precommit_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.697912 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Transition to ACTIVE for port object baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 will not be triggered until provisioned by entity L2. {{(pid=99546) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:19.842313 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.846207 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:19.882173 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-45851520-a0fe-4b86-a8c5-b12f24e1466e None None] Provisioning for port bef09144-4688-468a-b1ed-5f1bf3929ba3 completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:19.908364 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-45851520-a0fe-4b86-a8c5-b12f24e1466e None None] Provisioning complete for port bef09144-4688-468a-b1ed-5f1bf3929ba3 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:19.908989 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-45851520-a0fe-4b86-a8c5-b12f24e1466e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.935099 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.978047 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Attempting to bind port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:19.978429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Attempting to bind port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:19.979364 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Attempting to bind port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 on network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:19.981337 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23ed1fe9-6d65-4bc4-9ca7-8d147301610f None None] Pushing event updated for resources: {'Port': ['ID=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,revision_number=2']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:19.988400 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-23ed1fe9-6d65-4bc4-9ca7-8d147301610f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:19.991698 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:19.992084 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Checking segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99546) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:19.992495 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:19.992889 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Bound using segment: {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'} {{(pid=99546) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:19.993375 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.managers [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Bound port: baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '1a95fb4a-3292-44a5-abc6-7330179a94c6', 'network_type': 'flat', 'physical_network': 'physnet1', 'segmentation_id': None, 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc'}}] {{(pid=99546) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:19.993760 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:20.040671 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-45851520-a0fe-4b86-a8c5-b12f24e1466e None None] Port bef09144-4688-468a-b1ed-5f1bf3929ba3 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:20.087148 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] For port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99546) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:20.091588 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] For port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:20.202706 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:20.207317 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:20.287652 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:20.288287 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:20.316547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:20.350660 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-23ed1fe9-6d65-4bc4-9ca7-8d147301610f None None] Pushing event updated for resources: {'Port': ['ID=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,revision_number=3']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:20.365146 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-23ed1fe9-6d65-4bc4-9ca7-8d147301610f None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.158s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:20.409980 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:20.445694 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-f52e77b9-8007-4300-b81e-55b215178e49 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:20.475503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:20.538202 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Pushing event updated for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:20.540592 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_created-16269084', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._update_snat_v6_addrs_after_intf_update-491300'] for router_interface, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:20.546608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.103s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:20.640975 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-96c9e758-7ba8-4c26-a8c0-23603ac94a5b req-cda4e6b6-602a-490a-b0e2-2df5c12bda32 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:20.642494 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96c9e758-7ba8-4c26-a8c0-23603ac94a5b req-cda4e6b6-602a-490a-b0e2-2df5c12bda32 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 944 time: 0.2693546 May 25 16:13:20.725266 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-96c9e758-7ba8-4c26-a8c0-23603ac94a5b req-02552caf-5f72-406a-b2ef-5c0c0e1756a1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0766521 May 25 16:13:20.743074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:20.743074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:20.805955 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:20Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bef09144-4688-468a-b1ed-5f1bf3929ba3', 'name': '', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'mac_address': 'fa:16:3e:cd:26:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_interface', 'standard_attr_id': 670, 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:18Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 1, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'subnets': [{'id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:20.811819 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-100f9e16-f14c-42d8-b139-59b293c724a1 None None] The sync data for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi is: [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:20Z', 'revision_number': 3, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bef09144-4688-468a-b1ed-5f1bf3929ba3', 'name': '', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'mac_address': 'fa:16:3e:cd:26:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_interface', 'standard_attr_id': 670, 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:18Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 1, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'subnets': [{'id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:13:20.951544 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Router acd58228-b46f-4a8e-be41-0910702cff46 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:13:20.973603 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-994c80b8-1ce1-421d-bd25-58f6a9b5d961 None None] Provisioning complete for port bef09144-4688-468a-b1ed-5f1bf3929ba3 triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:20.974832 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-994c80b8-1ce1-421d-bd25-58f6a9b5d961 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:21.045475 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-994c80b8-1ce1-421d-bd25-58f6a9b5d961 None None] Port bef09144-4688-468a-b1ed-5f1bf3929ba3 cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:21.198964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:13:21.209401 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] 10.222.0.82 "PUT /v2.0/routers/acd58228-b46f-4a8e-be41-0910702cff46/add_router_interface HTTP/1.1" status: 200 len: 503 time: 3.3625882 May 25 16:13:21.933206 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:21.947197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3620ff0d-4973-47ad-a101-018f7092bb2f req-e682bd45-b06f-49de-bf55-9501eddbb18c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:21.955080 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3620ff0d-4973-47ad-a101-018f7092bb2f req-e682bd45-b06f-49de-bf55-9501eddbb18c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 944 time: 0.1062572 May 25 16:13:22.006440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3620ff0d-4973-47ad-a101-018f7092bb2f req-2cf22302-2bc3-47a8-a6b0-4daf3938cbc1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0502789 May 25 16:13:22.132105 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:22.164119 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:22.184963 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:22.185341 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.329278 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:22.332868 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.347479 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:22.347868 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.419388 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:13:22.439499 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:22.504425 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'code': 200} May 25 16:13:22.522909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.541649 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.544935 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d8a7809c-fb7c-4625-b0c2-889ac5a3e0f0 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:22.594501 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:22.641404 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:22.644653 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.100s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:22.676050 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:22.681702 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:22.784380 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:22.794519 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.113s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:23.236246 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Sync routers for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:13:23.272723 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:23.273704 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:23.326979 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1341a7c0-4699-4c7a-a3bd-a5635eb34536 req-9706deb2-b83f-4ec1-8d6f-390e0d4b1342 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:23.334477 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1341a7c0-4699-4c7a-a3bd-a5635eb34536 req-9706deb2-b83f-4ec1-8d6f-390e0d4b1342 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.1263361 May 25 16:13:23.374261 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-09540a06-9eeb-4b50-97aa-0e9f34f155a5 req-a9e3a3fc-aeff-48eb-a6b1-71ab8a706cf9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:23.375913 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-09540a06-9eeb-4b50-97aa-0e9f34f155a5 req-a9e3a3fc-aeff-48eb-a6b1-71ab8a706cf9 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1098 time: 0.4078710 May 25 16:13:23.420073 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1341a7c0-4699-4c7a-a3bd-a5635eb34536 req-e448de5e-c604-4e32-a751-703625acaba4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0842905 May 25 16:13:23.452155 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-09540a06-9eeb-4b50-97aa-0e9f34f155a5 req-ddca9510-1c2a-4d27-af67-3e57f8ddff1c service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0696874 May 25 16:13:23.683895 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:24.117113 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:22Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bef09144-4688-468a-b1ed-5f1bf3929ba3', 'name': '', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'mac_address': 'fa:16:3e:cd:26:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_interface', 'standard_attr_id': 670, 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': '', 'binding:vif_type': 'unbound', 'binding:vif_details': {}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:18Z', 'updated_at': '2021-05-25T16:13:20Z', 'revision_number': 2, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'subnets': [{'id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:24.126291 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Checking router: acd58228-b46f-4a8e-be41-0910702cff46 for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:13:24.126521 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Updating router acd58228-b46f-4a8e-be41-0910702cff46 port bef09144-4688-468a-b1ed-5f1bf3929ba3 binding host dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_port /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:205}} May 25 16:13:24.130633 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:24.207881 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:24.262151 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:24.263361 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:24.705303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f30581b6-6a01-4701-a6a6-b95b71470b1b req-0defb788-e186-424b-aa27-ec8a202044dc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:24.707293 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f30581b6-6a01-4701-a6a6-b95b71470b1b req-0defb788-e186-424b-aa27-ec8a202044dc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.1022840 May 25 16:13:24.773823 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f30581b6-6a01-4701-a6a6-b95b71470b1b req-1290441c-f4e7-41e5-92b8-28ad19a28801 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0604043 May 25 16:13:24.817121 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks [] for port, precommit_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:24.839549 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.provisioning_blocks [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Transition to ACTIVE for port object bef09144-4688-468a-b1ed-5f1bf3929ba3 will not be triggered until provisioned by entity L2. {{(pid=99547) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:24.931932 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:24.938994 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:25.300260 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59ed1428-46d7-4868-b5f4-ca5566b9b523 None None] Pushing event updated for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=3']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:25.306062 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:13:25.315569 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-59ed1428-46d7-4868-b5f4-ca5566b9b523 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.376s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:25.329126 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Attempting to bind port bef09144-4688-468a-b1ed-5f1bf3929ba3 on host dv-cinder-786384-6-iscsi for vnic_type normal with profile {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:25.329452 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Attempting to bind port bef09144-4688-468a-b1ed-5f1bf3929ba3 by drivers openvswitch,hyperv on host dv-cinder-786384-6-iscsi at level 0 using segments [{'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:25.329819 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Attempting to bind port bef09144-4688-468a-b1ed-5f1bf3929ba3 on network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:25.336882 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Checking agent: {'id': '4e42e5d6-292c-4383-ad94-dc558f1a54cb', 'agent_type': 'Open vSwitch agent', 'binary': 'neutron-openvswitch-agent', 'topic': 'N/A', 'host': 'dv-cinder-786384-6-iscsi', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 15, 54, 38), 'started_at': datetime.datetime(2021, 5, 25, 15, 55, 8), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 8), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'arp_responder_enabled': False, 'baremetal_smartnic': False, 'bridge_mappings': {'physnet1': 'br-data'}, 'datapath_type': 'system', 'devices': 0, 'enable_distributed_routing': False, 'extensions': [], 'in_distributed_mode': False, 'integration_bridge': 'br-int', 'l2_population': False, 'log_agent_heartbeats': False, 'ovs_capabilities': {'datapath_types': ['netdev', 'system'], 'iface_types': ['erspan', 'geneve', 'gre', 'internal', 'ip6erspan', 'ip6gre', 'lisp', 'patch', 'stt', 'system', 'tap', 'vxlan']}, 'ovs_hybrid_plug': False, 'resource_provider_bandwidths': {}, 'resource_provider_hypervisors': {'br-data': 'dv-cinder-786384-6-iscsi'}, 'resource_provider_inventory_defaults': {'allocation_ratio': 1.0, 'min_unit': 1, 'step_size': 1, 'reserved': 0}, 'tunnel_types': [], 'tunneling_ip': None, 'vhostuser_socket_dir': '/var/run/openvswitch'}, 'resource_versions': {'AddressGroup': '1.2', 'ConntrackHelper': '1.0', 'Log': '1.0', 'Network': '1.1', 'Port': '1.7', 'PortForwarding': '1.2', 'QosPolicy': '1.8', 'SecurityGroup': '1.4', 'SecurityGroupRule': '1.2', 'SubPort': '1.0', 'Subnet': '1.1', 'Trunk': '1.1'}} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:25.337506 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Checking segment: {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'} for mappings: {'physnet1': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99547) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:25.338585 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks [] for ovs_bridge_name, before_read {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:25.339030 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.drivers.mech_agent [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Bound using segment: {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'} {{(pid=99547) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:25.339536 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.managers [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Bound port: bef09144-4688-468a-b1ed-5f1bf3929ba3, host: dv-cinder-786384-6-iscsi, vif_type: ovs, vif_details: {"connectivity": "l2", "port_filter": true, "ovs_hybrid_plug": false, "datapath_type": "system", "bridge_name": "br-int"}, binding_levels: [{'bound_driver': 'openvswitch', 'bound_segment': {'id': '22130cb2-3662-4892-b32e-70b172a78d26', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 933, 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047'}}] {{(pid=99547) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:25.339929 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:25.476416 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] For port bef09144-4688-468a-b1ed-5f1bf3929ba3, host dv-cinder-786384-6-iscsi, cleared binding levels {{(pid=99547) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:25.480877 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.plugins.ml2.db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] For port bef09144-4688-468a-b1ed-5f1bf3929ba3, host dv-cinder-786384-6-iscsi, set binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bef09144-4688-468a-b1ed-5f1bf3929ba3,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99547) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:25.551454 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron_lib.callbacks.manager [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99547) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:25.556742 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-59ed1428-46d7-4868-b5f4-ca5566b9b523 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:25.645943 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:25.646590 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:25.704113 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-59ed1428-46d7-4868-b5f4-ca5566b9b523 None None] Pushing event updated for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=4']} {{(pid=99547) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:25.715335 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG oslo_concurrency.lockutils [None req-59ed1428-46d7-4868-b5f4-ca5566b9b523 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.159s {{(pid=99547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:25.912239 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:26.095896 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4ef13556-b848-4aa7-9b5c-a4ef45eac33b req-9a97f646-fb79-4bcf-aa31-8685e85be4ac tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:26.104931 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4ef13556-b848-4aa7-9b5c-a4ef45eac33b req-9a97f646-fb79-4bcf-aa31-8685e85be4ac tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.1573961 May 25 16:13:26.180598 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bef09144-4688-468a-b1ed-5f1bf3929ba3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:26.181221 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4ef13556-b848-4aa7-9b5c-a4ef45eac33b req-e881b0e1-1889-4cfa-9d9c-155fed5a2a39 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0726671 May 25 16:13:26.423784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8675fe2f-0224-4979-be4b-62404e841cb3 req-3ee77df0-30f4-4855-8883-aac910926071 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:26.424096 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning for port bef09144-4688-468a-b1ed-5f1bf3929ba3 completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:26.425834 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8675fe2f-0224-4979-be4b-62404e841cb3 req-3ee77df0-30f4-4855-8883-aac910926071 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.1210806 May 25 16:13:26.440786 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bef09144-4688-468a-b1ed-5f1bf3929ba3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:26.441199 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:26.527838 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8675fe2f-0224-4979-be4b-62404e841cb3 req-eee2ca34-4b9f-473e-84ad-f3858fc6a0ea tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0882401 May 25 16:13:26.558761 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51380) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:26.631026 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:22Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bef09144-4688-468a-b1ed-5f1bf3929ba3', 'name': '', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'mac_address': 'fa:16:3e:cd:26:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_interface', 'standard_attr_id': 670, 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:18Z', 'updated_at': '2021-05-25T16:13:25Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'subnets': [{'id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:26.640220 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-9823fac4-1b6d-429a-af9e-14f07c715792 None None] The sync data for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi is: [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:13:19Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:22Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': 'bef09144-4688-468a-b1ed-5f1bf3929ba3', 'name': '', 'network_id': '4ae9aef2-8f6b-4c56-b394-f86738d1b047', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'mac_address': 'fa:16:3e:cd:26:42', 'admin_state_up': True, 'status': 'DOWN', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_interface', 'standard_attr_id': 670, 'fixed_ips': [{'subnet_id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:18Z', 'updated_at': '2021-05-25T16:13:25Z', 'revision_number': 4, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'subnets': [{'id': 'd4ddf00b-460c-41c8-8051-1d52a3935d5b', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:13:26.674303 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:26.699854 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-8c12ef1d-dc5a-43bc-9f0a-e8b7e42e3bfd tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:26.703439 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8c12ef1d-dc5a-43bc-9f0a-e8b7e42e3bfd tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1093 time: 0.1414273 May 25 16:13:26.708766 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51384) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:13:26.728029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address': '172.20.0.5'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:26.745720 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:26.751145 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] For port bef09144-4688-468a-b1ed-5f1bf3929ba3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bef09144-4688-468a-b1ed-5f1bf3929ba3,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:26.756119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:floatingip and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:26.761305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for floatingip was recalculated. Used quota:1. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:26.777524 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource floatingip. Total usage: 1; quota limit: 50; headroom:49 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:26.840258 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:26.842545 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-23ed1fe9-6d65-4bc4-9ca7-8d147301610f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:26.859208 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:26.994246 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Pushing event updated for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:26.998164 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.156s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:27.104515 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:27.492093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.27.2.21 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/3c840498-7e4d-4fb0-bf72-4d3d2dfe855d) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:27.715487 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:27.831464 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:27.838862 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d12a97cf-9876-4a55-96ad-270fe65eb745 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:27.910953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:27.914772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:27.920260 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:27.982272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=3c840498-7e4d-4fb0-bf72-4d3d2dfe855d,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:27.985171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.146s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:28.165685 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bef09144-4688-468a-b1ed-5f1bf3929ba3 up at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:13:28.304997 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Provisioning complete for port bef09144-4688-468a-b1ed-5f1bf3929ba3 triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:28.305328 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:28.598357 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:28.806445 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:29.140144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:29.231922 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:29.240370 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:29.387870 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=3c840498-7e4d-4fb0-bf72-4d3d2dfe855d,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:29.393065 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.153s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:29.401454 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:29.613607 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP 6b0ddb8d-7792-4174-bb8b-673891182098 associated. External IP: 172.27.2.21, port: 4b64460a-d17d-4ac3-82d0-d9990d9be8b4. May 25 16:13:30.108025 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:13:30.373902 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:13:30.393220 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:13:30.421660 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:30.422728 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:30.502429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:30.606988 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 3.8863258 May 25 16:13:30.606988 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:13:30.690073 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'status': 'completed', 'code': 200} May 25 16:13:30.809543 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:30.852710 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-eb78a614-9c9b-4fd9-b47b-700772503231 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:30.856160 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-eb78a614-9c9b-4fd9-b47b-700772503231 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1098 time: 0.1201909 May 25 16:13:31.110624 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-288b64b3-b1ea-4c6b-b98a-4a00620f5c78 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:31.113376 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-288b64b3-b1ea-4c6b-b98a-4a00620f5c78 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.2506399 May 25 16:13:31.277246 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-8c5ac0eb-f100-4f69-940b-0dd69e02ac96 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 998 time: 0.1575682 May 25 16:13:31.390140 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-d226b6db-446e-43cd-8ca9-840fa419933f service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:31.392648 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-d226b6db-446e-43cd-8ca9-840fa419933f service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.1090512 May 25 16:13:31.416758 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:28Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:31.422454 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:13:31.435462 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:13:31.436557 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 0, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:13:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:31.545927 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-63d8e6ad-b108-414e-a149-fd094a0910fc service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:31.545927 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-63d8e6ad-b108-414e-a149-fd094a0910fc service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1457810 May 25 16:13:31.558827 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-f6cf558b-f835-4a7b-9180-d7228b77e2e7 service neutron] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 16:13:31.560760 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-f6cf558b-f835-4a7b-9180-d7228b77e2e7 service neutron] 10.222.0.42 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0084093 May 25 16:13:31.764248 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:13:31.829581 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-b72637e0-6652-4e34-977c-072c748163e5 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2618971 May 25 16:13:32.153464 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83aaeb95-35c8-4dcc-9460-4a68bf602902 req-7c6f816e-b666-47b6-bfe2-adf44f362885 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.3150544 May 25 16:13:32.381104 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:28Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:13:32.386822 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-1e0495d3-7094-4947-ad00-da40c8d62bd4 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'DOWN', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:28Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:13:32.665565 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4aa4d9d7-91c1-4c6e-9262-83e75728e8a1 None None] New status for floating IP 6b0ddb8d-7792-4174-bb8b-673891182098: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:13:35.646972 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4420a91d-e5c0-4823-9fd9-b511abd53bb8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:35.968867 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-540054a5-571c-45c4-8f81-401aefeb1c32 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:35.969564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-540054a5-571c-45c4-8f81-401aefeb1c32 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Synchronizing usage tracker for tenant:7413f95c7d284674bb0d587de6b42810 on resource:network {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:13:36.082041 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-540054a5-571c-45c4-8f81-401aefeb1c32 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Unset dirty status for tenant:7413f95c7d284674bb0d587de6b42810 on resource:network {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:13:36.083683 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-540054a5-571c-45c4-8f81-401aefeb1c32 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 815 time: 0.2866542 May 25 16:13:36.105585 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-752d38e1-bed6-4624-88c2-4311cc10bcff tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/quotas/7413f95c7d284674bb0d587de6b42810 HTTP/1.1" status: 200 len: 375 time: 0.0144901 May 25 16:13:36.213444 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c6750322-777f-4c4c-a17f-3a0d28000dcd tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&fields=id HTTP/1.1" status: 200 len: 302 time: 0.1011887 May 25 16:13:36.231518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-9417ee0d-816b-427d-bab1-15b96b6d1ec3 service neutron] http://10.222.0.82:9696/v2.0/extensions returned with HTTP 200 {{(pid=99545) __call__ /opt/stack/neutron/neutron/wsgi.py:729}} May 25 16:13:36.232416 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-9417ee0d-816b-427d-bab1-15b96b6d1ec3 service neutron] 10.222.0.82 "GET /v2.0/extensions HTTP/1.1" status: 200 len: 13640 time: 0.0056524 May 25 16:13:36.498029 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-cecc144b-543b-4bc5-bb2f-c34d1721b3b1 service neutron] 10.222.0.82 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2547085 May 25 16:13:36.742237 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c07eddcc-69b9-4780-84b1-d6e4114fad23 service neutron] 10.222.0.82 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2353778 May 25 16:13:37.102978 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-79e2b693-1f38-482b-9a33-ab7dc3b95d08 req-3c3e81fe-68a3-489b-9337-3a141bd9e6fe tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 210 time: 0.0458403 May 25 16:13:38.161411 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:38.219483 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3931e539-8222-4bce-a209-efeee14a45fa req-e71c9214-5dd4-41e3-a7bf-c0925e262b9e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 210 time: 0.0479903 May 25 16:13:38.439220 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-fb2916b5-8323-44ee-a478-3dc9adbd71fb None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:39.207008 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:39.390980 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6cb9420b-7145-4782-817b-793bd7750bf8 req-1201ca97-ee98-44a4-a2ea-f520e2eacc67 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 210 time: 0.0579906 May 25 16:13:40.518505 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-1c35c317-2aa6-4a9e-8c6b-0f635aac0e3e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:40.520542 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-1c35c317-2aa6-4a9e-8c6b-0f635aac0e3e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 815 time: 0.2007864 May 25 16:13:40.574400 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'port': {'device_id': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'admin_state_up': True, 'tenant_id': '7413f95c7d284674bb0d587de6b42810'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:40.598132 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ad62274a-b035-420f-bf77-64402d6e78cc req-5cd42aed-5ea1-48c2-a824-d94917d44414 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 210 time: 0.0654185 May 25 16:13:40.618668 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:40.739943 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:13:40.743837 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:port and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:13:40.748005 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for port was recalculated. Used quota:2. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:13:40.753398 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource port. Total usage: 2; quota limit: 500; headroom:498 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:13:40.861765 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Made reservation on behalf of 7413f95c7d284674bb0d587de6b42810 for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:13:40.865559 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:41.106420 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Allocated IP 172.20.0.4 (ceb9e841-143d-468e-800d-756d578d900d/70734104-38bb-429b-abc3-63fbc11e4122/145301a2-46f6-4ded-b871-52227a79d41a) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:13:41.144743 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:41.385357 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:41.404357 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Transition to ACTIVE for port object 145301a2-46f6-4ded-b871-52227a79d41a will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:41.555822 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:41.562404 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9d9a7495-28bc-494b-8e95-3eccf6f621aa tempest-ServerRescueNegativeTestJSON-1571698249 tempest-ServerRescueNegativeTestJSON-1571698249-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:41.647034 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:41.652931 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.090s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:41.663733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Network ceb9e841-143d-468e-800d-756d578d900d is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:13:41.665190 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:13:41.668857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:13:41.765636 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:41.768220 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:41.939240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-003a0a99-453a-4b79-8ac4-6dad67fa30a1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:41.940012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-003a0a99-453a-4b79-8ac4-6dad67fa30a1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Synchronizing usage tracker for tenant:7413f95c7d284674bb0d587de6b42810 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:13:41.944421 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.3790910 May 25 16:13:42.041106 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-003a0a99-453a-4b79-8ac4-6dad67fa30a1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Unset dirty status for tenant:7413f95c7d284674bb0d587de6b42810 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:13:42.042737 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-003a0a99-453a-4b79-8ac4-6dad67fa30a1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.2537155 May 25 16:13:42.103821 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-301fbadc-4e45-4b39-9a98-c0569f1798eb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Synchronizing usage tracker for tenant:7413f95c7d284674bb0d587de6b42810 on resource:security_group {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:13:42.253240 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-301fbadc-4e45-4b39-9a98-c0569f1798eb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Unset dirty status for tenant:7413f95c7d284674bb0d587de6b42810 on resource:security_group {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:13:42.257082 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ce2ca5b8-cca9-4beb-ae88-4036567e2aaa req-301fbadc-4e45-4b39-9a98-c0569f1798eb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1983724 May 25 16:13:42.281102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-363c1ad1-1208-46e4-a9fc-ade429b2097e None None] Provisioning for port 145301a2-46f6-4ded-b871-52227a79d41a completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:13:42.287218 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-363c1ad1-1208-46e4-a9fc-ade429b2097e None None] Provisioning complete for port 145301a2-46f6-4ded-b871-52227a79d41a triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:13:42.288022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-363c1ad1-1208-46e4-a9fc-ade429b2097e None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:42.372390 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-363c1ad1-1208-46e4-a9fc-ade429b2097e None None] Port 145301a2-46f6-4ded-b871-52227a79d41a cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:13:43.467740 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c3591bb6-fd11-430f-b1df-5bd5b5d86eda req-fd4a4259-90aa-4acc-b9f3-0fddc2af463f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:43.469237 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3591bb6-fd11-430f-b1df-5bd5b5d86eda req-fd4a4259-90aa-4acc-b9f3-0fddc2af463f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.0861971 May 25 16:13:43.536860 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c3591bb6-fd11-430f-b1df-5bd5b5d86eda req-2737c268-fc17-4db6-89e1-123a432eb4b1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0580084 May 25 16:13:44.789880 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-f0ea542e-7e11-4594-acca-d68ae03b76f1 req-a40780c2-cca1-4e30-8d20-947bf581bfff tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:44.791447 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0ea542e-7e11-4594-acca-d68ae03b76f1 req-a40780c2-cca1-4e30-8d20-947bf581bfff tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.0979028 May 25 16:13:44.842126 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-f0ea542e-7e11-4594-acca-d68ae03b76f1 req-49d552e5-bb87-4acc-997e-2d9a13ae472d tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0446815 May 25 16:13:46.058829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8447c9ac-1b10-4b65-ab12-46bdb2feb00a req-3925ae1e-2f96-4e28-b85a-401071b95a91 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:46.060427 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8447c9ac-1b10-4b65-ab12-46bdb2feb00a req-3925ae1e-2f96-4e28-b85a-401071b95a91 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.0910213 May 25 16:13:46.122508 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8447c9ac-1b10-4b65-ab12-46bdb2feb00a req-6ae944dc-41bf-4894-b0fd-e1ffcf03e747 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0555713 May 25 16:13:47.337806 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0529cb09-32b2-42f5-b70a-3b238ddeaaf3 req-fabcb843-6b2f-4b44-b2f6-6f614b3b6099 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:47.339473 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0529cb09-32b2-42f5-b70a-3b238ddeaaf3 req-fabcb843-6b2f-4b44-b2f6-6f614b3b6099 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.0872920 May 25 16:13:47.393720 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0529cb09-32b2-42f5-b70a-3b238ddeaaf3 req-df184949-f84d-4bde-8144-03ad850197dd tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0477431 May 25 16:13:48.734332 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0f4911aa-6e98-479a-aea6-851f802b7f83 req-87584c60-db0e-4db1-984f-8bbfd2cedf5a tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:48.747492 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0f4911aa-6e98-479a-aea6-851f802b7f83 req-87584c60-db0e-4db1-984f-8bbfd2cedf5a tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.2351580 May 25 16:13:48.814058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0f4911aa-6e98-479a-aea6-851f802b7f83 req-2051a53d-444c-4243-999c-169889a38f42 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0653710 May 25 16:13:50.054472 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-702ad6ec-379c-4118-845b-dd2f27f3859e req-5a09725b-5db9-420c-8840-8a3a68c3a1fd tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:50.056075 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-702ad6ec-379c-4118-845b-dd2f27f3859e req-5a09725b-5db9-420c-8840-8a3a68c3a1fd tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.1214912 May 25 16:13:50.115716 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-702ad6ec-379c-4118-845b-dd2f27f3859e req-f298b3d7-5c60-46da-931c-0f8ec5097bc5 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0509248 May 25 16:13:51.328679 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1bde0107-8e2e-4e27-ab15-2bd198bea7f3 req-756efc3b-f6f6-498c-be63-9c9168f8ba67 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:51.330213 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1bde0107-8e2e-4e27-ab15-2bd198bea7f3 req-756efc3b-f6f6-498c-be63-9c9168f8ba67 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.0922043 May 25 16:13:51.381121 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1bde0107-8e2e-4e27-ab15-2bd198bea7f3 req-ed61f035-93d0-4132-b86b-6eaef4183282 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0424898 May 25 16:13:52.181994 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Request body: {'port': {'device_id': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:13:52.306548 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:52.676003 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7cc3d38e-5c84-4f48-afb1-278edd13df7b req-910913e3-ac2c-4319-80de-34829e2b3c64 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:52.677459 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7cc3d38e-5c84-4f48-afb1-278edd13df7b req-910913e3-ac2c-4319-80de-34829e2b3c64 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 931 time: 0.1839263 May 25 16:13:52.782829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id 145301a2-46f6-4ded-b871-52227a79d41a {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:13:52.793736 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7cc3d38e-5c84-4f48-afb1-278edd13df7b req-00279545-ee07-4019-9c75-da0f051ea5a2 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.1036956 May 25 16:13:52.795706 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:52.816345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Transition to ACTIVE for port object 145301a2-46f6-4ded-b871-52227a79d41a will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:13:52.890447 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:52.894665 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1ab8d8b9-e6c7-4e31-be54-ad206682220b tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:53.040458 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:53.068094 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.173s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:53.161901 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Attempting to bind port 145301a2-46f6-4ded-b871-52227a79d41a on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:13:53.162328 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Attempting to bind port 145301a2-46f6-4ded-b871-52227a79d41a by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:13:53.162658 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Attempting to bind port 145301a2-46f6-4ded-b871-52227a79d41a on network ceb9e841-143d-468e-800d-756d578d900d {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:53.169241 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Port 145301a2-46f6-4ded-b871-52227a79d41a on network ceb9e841-143d-468e-800d-756d578d900d not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:13:53.169687 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Attempting to bind port 145301a2-46f6-4ded-b871-52227a79d41a on network ceb9e841-143d-468e-800d-756d578d900d {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:13:53.177606 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 13, 40), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:13:53.177848 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Checking segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:13:53.178239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Bound using segment: {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:13:53.178693 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Bound port: 145301a2-46f6-4ded-b871-52227a79d41a, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': '6075f059-4c81-4008-876b-49bbdcfe295e', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 916, 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:13:53.179084 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:53.277572 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:13:53.280789 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:13:53.359772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:53.363949 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:53.441427 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:53.441969 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:53.443408 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] 10.222.0.42 "PUT /v2.0/ports/145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 1093 time: 1.2693868 May 25 16:13:53.444737 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:13:53.465888 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:53.469992 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca55fb94-2ca9-41e4-a24c-8229d3cb16b6 service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.106s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:53.504809 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:13:53.661944 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-05ea20c2-dcc9-4157-b61a-a8ebc6125346 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:53.665263 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-05ea20c2-dcc9-4157-b61a-a8ebc6125346 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1096 time: 0.1112902 May 25 16:13:53.838330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-d03e7a27-9f1c-4fc9-9d6a-945e407bba15 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:53.840526 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-d03e7a27-9f1c-4fc9-9d6a-945e407bba15 service neutron] 10.222.0.42 "GET /v2.0/networks?tenant_id=7413f95c7d284674bb0d587de6b42810&shared=False HTTP/1.1" status: 200 len: 916 time: 0.1639137 May 25 16:13:54.166531 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-bbcb4f0c-6419-4b69-a5a4-25a97ce4a4d4 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:54.168268 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ca1aadcb-b9fd-4ac4-a27d-3f89edc15841 req-bbcb4f0c-6419-4b69-a5a4-25a97ce4a4d4 service neutron] 10.222.0.42 "GET /v2.0/networks?shared=True HTTP/1.1" status: 200 len: 870 time: 0.3215370 May 25 16:13:54.191952 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-38b3258f-212e-4aa4-a64f-2a43496920d5 req-3f806f45-e1fd-43f8-b600-0222d586f154 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:54.194485 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-38b3258f-212e-4aa4-a64f-2a43496920d5 req-3f806f45-e1fd-43f8-b600-0222d586f154 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 943 time: 0.2616365 May 25 16:13:54.290625 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-38b3258f-212e-4aa4-a64f-2a43496920d5 req-ad4d98af-8b6a-4129-bef3-08957ebc05fc tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0898023 May 25 16:13:54.331749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-d8f5df29-34fc-44aa-ac17-be7915072dae service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:54.334302 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-d8f5df29-34fc-44aa-ac17-be7915072dae service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1096 time: 0.1016614 May 25 16:13:54.444754 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-adfbec87-888e-4f8a-9eb1-dc9e776fc334 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 217 time: 0.0975144 May 25 16:13:54.518781 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ef40da3f-5a69-42a8-aa69-1d353b07bfbd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:54.520447 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-ef40da3f-5a69-42a8-aa69-1d353b07bfbd service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0697291 May 25 16:13:54.666104 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-d71201ab-212d-44ba-b6df-23663736353c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:54.669203 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-d71201ab-212d-44ba-b6df-23663736353c service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1413026 May 25 16:13:54.818058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-e26c2a14-03a4-423e-83bb-cc9b3010c8c8 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1421921 May 25 16:13:54.987518 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75a0cae7-0ceb-4c10-82ca-2db0f503728e req-21df10b7-8521-4c5b-bb62-7c856aaec9fc service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1627712 May 25 16:13:55.708324 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-899abff3-4538-4bd7-8e17-94afad58582e req-f444e2f5-9234-4ccd-9c1a-a56defde6d03 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:55.708324 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-899abff3-4538-4bd7-8e17-94afad58582e req-f444e2f5-9234-4ccd-9c1a-a56defde6d03 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 943 time: 0.1387568 May 25 16:13:55.708324 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-899abff3-4538-4bd7-8e17-94afad58582e req-0a066861-be1b-4f11-bf5d-c82c3ae28a5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0715094 May 25 16:13:56.903990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-94acf4ca-ca94-47bf-9b57-8e74806bf284 req-d02cfb9b-4517-42a8-89ff-64a7e6944ea4 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:56.906606 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94acf4ca-ca94-47bf-9b57-8e74806bf284 req-d02cfb9b-4517-42a8-89ff-64a7e6944ea4 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 943 time: 0.1011372 May 25 16:13:56.958831 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-94acf4ca-ca94-47bf-9b57-8e74806bf284 req-39337589-f5ce-4522-bc77-273a073a5633 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0453963 May 25 16:13:58.171871 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 145301a2-46f6-4ded-b871-52227a79d41a details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:13:58.311506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7fbea499-a7df-4d04-9205-dc778ce2340a req-4e987aaf-f363-46db-804e-8baa3ce85d3d tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:58.314092 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7fbea499-a7df-4d04-9205-dc778ce2340a req-4e987aaf-f363-46db-804e-8baa3ce85d3d tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 943 time: 0.1423028 May 25 16:13:58.377281 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7fbea499-a7df-4d04-9205-dc778ce2340a req-ada479c0-607b-41fc-acc8-d0cee3a4b84e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0523620 May 25 16:13:58.417741 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:58.435820 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': '145301a2-46f6-4ded-b871-52227a79d41a', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'port_id': '145301a2-46f6-4ded-b871-52227a79d41a', 'mac_address': 'fa:16:3e:74:9f:b4', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 916, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:13:58.495808 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:58.516867 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id 145301a2-46f6-4ded-b871-52227a79d41a {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:13:58.537445 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:13:58.606939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:13:58.609734 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-83c0da92-5b2f-4137-9d5c-1499285fb1fd None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:13:58.689531 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:13:58.700429 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.700773 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.701229 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.701769 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.702207 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.702549 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.702928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.703314 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.703657 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.704197 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.704623 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.704985 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:13:58.710826 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.101s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:13:58.885190 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:13:59.620760 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d5f9ffd5-efd6-48bb-b49a-6c276616317a req-5853a10f-6bad-425b-9d8c-15f56f4d4c18 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:13:59.622372 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5f9ffd5-efd6-48bb-b49a-6c276616317a req-5853a10f-6bad-425b-9d8c-15f56f4d4c18 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 944 time: 0.1170368 May 25 16:13:59.675905 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d5f9ffd5-efd6-48bb-b49a-6c276616317a req-a840345f-77ca-44ea-8fa4-e4786fb59a8e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0441313 May 25 16:14:00.182226 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 145301a2-46f6-4ded-b871-52227a79d41a up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:14:00.353356 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port 145301a2-46f6-4ded-b871-52227a79d41a completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:14:00.360272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port 145301a2-46f6-4ded-b871-52227a79d41a triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:14:00.360703 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:00.508857 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:00.549290 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:14:00.560386 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:14:00.662596 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:00.665639 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-03582111-ad4b-4da3-a4bc-d746c0a1b33f None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:00.792589 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'code': 200} May 25 16:14:00.799905 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:00.804325 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.139s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:00.938672 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-8bc1ce22-dbb1-4f40-a7d2-4f6614f85452 req-87091de4-f8d2-412a-ab09-b3838939dc61 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:00.941432 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8bc1ce22-dbb1-4f40-a7d2-4f6614f85452 req-87091de4-f8d2-412a-ab09-b3838939dc61 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.1081486 May 25 16:14:01.011058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-8bc1ce22-dbb1-4f40-a7d2-4f6614f85452 req-ae933f32-c00d-4b63-a937-d5bf0242acc8 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0625682 May 25 16:14:01.212209 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4c3047c4-b354-4930-89ca-f179c3a600ad req-f7bf0cb1-8fce-40d6-b75d-92f50d852c80 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:01.215237 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4c3047c4-b354-4930-89ca-f179c3a600ad req-f7bf0cb1-8fce-40d6-b75d-92f50d852c80 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1098 time: 0.0976012 May 25 16:14:01.261980 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4c3047c4-b354-4930-89ca-f179c3a600ad req-5f2d273b-50db-49d4-83ed-cfd3d81e551d service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0403864 May 25 16:14:02.225332 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25896c37-0042-45b4-9804-d03b2e96b112 req-30dba5ec-7fab-42e9-b3da-28b7f0e6f88a tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:02.228157 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25896c37-0042-45b4-9804-d03b2e96b112 req-30dba5ec-7fab-42e9-b3da-28b7f0e6f88a tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0930982 May 25 16:14:02.282077 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25896c37-0042-45b4-9804-d03b2e96b112 req-b67acc6c-b51e-4394-be5f-181930fdde1c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0435367 May 25 16:14:02.302186 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:14:03.490951 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-859d85a6-bac4-417f-9adb-aeb5d61abce4 req-e8849b90-e227-4fac-a068-13d598fcc159 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:03.492737 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-859d85a6-bac4-417f-9adb-aeb5d61abce4 req-e8849b90-e227-4fac-a068-13d598fcc159 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0869446 May 25 16:14:03.548615 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-859d85a6-bac4-417f-9adb-aeb5d61abce4 req-908617f6-79a7-4df0-a7ec-5722a4a3cc0b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0464244 May 25 16:14:03.754466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e9ea426b-6575-43f3-9916-636c14a856c5 req-06c39dad-0c9e-43cb-a8f0-d92566d8d5c5 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:03.756089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e9ea426b-6575-43f3-9916-636c14a856c5 req-06c39dad-0c9e-43cb-a8f0-d92566d8d5c5 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0951066 May 25 16:14:03.809567 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e9ea426b-6575-43f3-9916-636c14a856c5 req-1496ce90-9c1c-4a23-8c77-989e82ad8ada tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0439680 May 25 16:14:04.338462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-16e41db2-33da-4e82-9b1a-1d5426c1bc37 req-1a1c7429-4f3e-46ad-aafc-1572dc949334 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:04.340559 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-16e41db2-33da-4e82-9b1a-1d5426c1bc37 req-1a1c7429-4f3e-46ad-aafc-1572dc949334 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0915699 May 25 16:14:04.388875 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-16e41db2-33da-4e82-9b1a-1d5426c1bc37 req-dcc61cba-44ad-4112-b3ff-ab4b3ecfc340 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0421951 May 25 16:14:05.715079 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b52806f9-7e69-459e-98ce-936d953d62a1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:08.406388 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:08.497876 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b070d69b-2fb4-4d03-9483-701fbdf0b988 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:09.125949 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:10.565885 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:11.412031 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:14.028798 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-d81f7819-36a0-4a1b-b5d3-ff7911095100 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0725811 May 25 16:14:14.309166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-4cfc3ae4-37a8-4126-99b5-9570c4d2fe9c service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:14.310639 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-4cfc3ae4-37a8-4126-99b5-9570c4d2fe9c service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1098 time: 0.1822424 May 25 16:14:14.468649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-c459be6e-e0c5-4883-85e1-f7891f061da8 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:14.470079 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-c459be6e-e0c5-4883-85e1-f7891f061da8 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1327026 May 25 16:14:14.584898 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-a808bddb-9d69-4028-843c-495d7816a42e service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 1000 time: 0.1049941 May 25 16:14:14.654257 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-b011c820-f6ac-47ea-8b1a-e93a554ebd93 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:14.655756 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-b011c820-f6ac-47ea-8b1a-e93a554ebd93 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0624526 May 25 16:14:14.741326 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-d74db182-5877-45d0-b6ea-82c66662b887 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:14.742747 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-d74db182-5877-45d0-b6ea-82c66662b887 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0801110 May 25 16:14:14.891954 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-5d7ec126-cc94-4ff2-9026-b29a5a87606d service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1362967 May 25 16:14:15.036892 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a7b3673e-1540-4726-8c27-7de32b812c27 req-328b394d-d533-4923-9d3f-5081f7479195 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1282840 May 25 16:14:17.685719 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-97d116d4-52be-4966-9eb0-ce13da835987 req-a37cdac9-4849-453a-b85a-798440b64b04 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:17.687234 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97d116d4-52be-4966-9eb0-ce13da835987 req-a37cdac9-4849-453a-b85a-798440b64b04 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0934050 May 25 16:14:17.734708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-97d116d4-52be-4966-9eb0-ce13da835987 req-675e7bde-2664-44ef-ac42-44ef0ff0aad7 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0416999 May 25 16:14:17.766029 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51572) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:17.890407 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-5439a10b-707c-493b-b30f-d9de3b16f2f2 tempest-TestMinimumBasicScenario-984417978 tempest-TestMinimumBasicScenario-984417978-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:17.891027 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5439a10b-707c-493b-b30f-d9de3b16f2f2 tempest-TestMinimumBasicScenario-984417978 tempest-TestMinimumBasicScenario-984417978-project-admin] Synchronizing usage tracker for tenant:92165e91edbb47c88928d15e0da2d7bd on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:14:17.994547 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-5439a10b-707c-493b-b30f-d9de3b16f2f2 tempest-TestMinimumBasicScenario-984417978 tempest-TestMinimumBasicScenario-984417978-project-admin] Unset dirty status for tenant:92165e91edbb47c88928d15e0da2d7bd on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:14:17.995825 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5439a10b-707c-493b-b30f-d9de3b16f2f2 tempest-TestMinimumBasicScenario-984417978 tempest-TestMinimumBasicScenario-984417978-project-admin] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1093 time: 0.2287619 May 25 16:14:17.999650 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51578) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:18.005221 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'port_id': '145301a2-46f6-4ded-b871-52227a79d41a', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'fixed_ip_address': '172.20.0.4'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:18.011716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:18.015834 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:floatingip and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:18.019179 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for floatingip was recalculated. Used quota:0. {{(pid=99545) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:18.025494 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource floatingip. Total usage: 0; quota limit: 50; headroom:50 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:18.109247 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for floatingip, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:18.276530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:18.473239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Allocated IP 172.27.2.185 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/df27303f-a70f-49d6-8356-b5f8d0e19eaa) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:14:18.709619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:18.909724 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:18.915100 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-ca1c9b33-cd6c-4b65-9480-b1f0b766bd77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:18.997269 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:14:19.000691 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:14:19.004725 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:14:19.036248 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'Port': ['ID=df27303f-a70f-49d6-8356-b5f8d0e19eaa,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:19.057453 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.057871 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.058308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.058683 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.059068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.059446 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.059802 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.060154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.060504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.060857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.061204 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.061552 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:19.067060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.152s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:19.565685 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:19.727201 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:19.945282 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:20.034122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:20.038033 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a8d85e65-fd8b-4ec2-a999-6e78d06bcad1 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:20.148488 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'Port': ['ID=df27303f-a70f-49d6-8356-b5f8d0e19eaa,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:20.152627 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.115s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:20.160843 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:20.340464 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Floating IP c4abf34e-d7c2-4615-a16a-a8c322bada22 associated. External IP: 172.27.2.185, port: 145301a2-46f6-4ded-b871-52227a79d41a. May 25 16:14:20.743784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Router 0492512e-ccab-4c81-9788-1e5b615f9dc3 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:14:20.966662 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:14:20.979538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Sync routers for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:14:20.992221 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:14:20.992903 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:14:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:14:21.108564 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.168662 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 996 time: 3.1678236 May 25 16:14:21.169416 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:14:21.171473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51582) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:21.186166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group': {'name': 'tempest-secgroup-smoke-1667647037', 'description': 'tempest-secgroup-smoke-1667647037 description', 'project_id': '7413f95c7d284674bb0d587de6b42810'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:21.187783 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:14:21.192989 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:21.203213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group. Total usage: 1; quota limit: 100; headroom:99 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:21.218597 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:14:21.303230 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.333572 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:21.337783 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:21.341751 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:4. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:21.349860 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 2 items for resource security_group_rule. Total usage: 4; quota limit: 100; headroom:96 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:21.366338 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-ec041baf-45e2-4634-870f-c6717aa2ec54 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:21.366338 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-ec041baf-45e2-4634-870f-c6717aa2ec54 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1098 time: 0.0917275 May 25 16:14:21.446897 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.571638 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.576351 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-372014a2-b13f-4757-922f-b43ccac180c9 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:21.622231 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroup': ['ID=134145c7-9f57-4f74-a5b6-0e989035223a,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:21.630933 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.631330 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.631651 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.632020 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.632379 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.632742 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.633144 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.633501 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.633870 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.634220 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.634573 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.634945 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:14:21.639417 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.063s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:21.652690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-df74ba46-affb-4add-902d-5d0c59276430 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:21.654416 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-df74ba46-affb-4add-902d-5d0c59276430 service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.2859089 May 25 16:14:21.684859 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'c4abf34e-d7c2-4615-a16a-a8c322bada22', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'floating_ip_address': '172.27.2.185', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'port_id': '145301a2-46f6-4ded-b871-52227a79d41a', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 675, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'mac_address': 'fa:16:3e:74:9f:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:14:19Z', 'updated_at': '2021-05-25T16:14:19Z', 'revision_number': 0, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:14:21.689351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Checking router: 0492512e-ccab-4c81-9788-1e5b615f9dc3 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:14:21.697422 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.699176 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-45155208-9212-4a67-939b-09265e197520 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-groups HTTP/1.1" status: 201 len: 1794 time: 0.5261314 May 25 16:14:21.700231 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:14:21.700893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:14:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:14:21.702117 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51588) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:21.709252 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:21.716077 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:21.721050 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:21.725224 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:6. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:21.732366 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 6; quota limit: 100; headroom:94 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:21.781320 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-fb96bac4-0f0a-4b7b-9de9-954458aa6b08 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 999 time: 0.1187878 May 25 16:14:21.823276 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.869195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-8e2cb369-ec83-4a2a-bf76-6377efcae64a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:21.872403 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-8e2cb369-ec83-4a2a-bf76-6377efcae64a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0847380 May 25 16:14:21.901195 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:21.923557 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:14:21.962619 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-24acb3f8-ef17-41c1-90e6-40adaf34a179 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:21.964239 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-24acb3f8-ef17-41c1-90e6-40adaf34a179 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0850251 May 25 16:14:22.016912 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.019630 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-24779b6b-b355-493b-a433-5b11ff1a0ae9 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:22.044748 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=d20770ca-0650-4a40-b9c7-cbe1f0d228d4,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:22.054557 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:22.118964 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-328d2766-0be5-40b1-b57c-e26c03cfb534 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1446872 May 25 16:14:22.132249 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.139108 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-156e34c6-7d29-40d6-af36-36417b7d9eeb tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 764 time: 0.4309185 May 25 16:14:22.139418 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51592) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:22.145409 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'tcp', 'port_range_min': 22, 'port_range_max': 22, 'direction': 'egress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:22.152246 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:22.157156 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:22.161122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:7. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:22.171166 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 7; quota limit: 100; headroom:93 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:22.294134 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.348218 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0b79d9dd-e645-46a3-9455-98ba553c0c06 req-6e32d3c5-2ff7-46bc-974b-1b544f14f23f service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2209530 May 25 16:14:22.376657 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.395097 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'c4abf34e-d7c2-4615-a16a-a8c322bada22', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'floating_ip_address': '172.27.2.185', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'port_id': '145301a2-46f6-4ded-b871-52227a79d41a', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 675, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'mac_address': 'fa:16:3e:74:9f:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:14:19Z', 'updated_at': '2021-05-25T16:14:19Z', 'revision_number': 0, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:14:22.396819 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ae2c9c3a-6aa4-4529-baed-3380b59e0551 None None] The sync data for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi is: [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': 'c4abf34e-d7c2-4615-a16a-a8c322bada22', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'floating_ip_address': '172.27.2.185', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'port_id': '145301a2-46f6-4ded-b871-52227a79d41a', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 675, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'mac_address': 'fa:16:3e:74:9f:b4', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:14:19Z', 'updated_at': '2021-05-25T16:14:19Z', 'revision_number': 0, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:14:22.455947 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.458235 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d9249193-51a9-4fd8-93bf-b0edd4aa4ff9 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:22.484688 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=f4a9fb29-385a-44d2-a5aa-770185286c93,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:22.491308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.033s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:22.571421 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.572419 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-52228e77-ed72-4683-a917-d2163a4de71b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 763 time: 0.4319835 May 25 16:14:22.575489 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51596) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:22.581272 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'icmp', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:22.587721 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:22.592061 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:22.596303 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:8. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:22.602430 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 8; quota limit: 100; headroom:92 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:22.612286 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-cdee3272-baed-4d5a-b84f-6aa58acb5479 None None] New status for floating IP c4abf34e-d7c2-4615-a16a-a8c322bada22: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:14:22.679867 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.756998 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.840561 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.842805 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:22.867463 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=8ee8d162-63bd-4f84-9113-2cf6883969de,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:22.869914 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:22.948640 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:22.949862 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-889c59eb-cf91-4f65-94dc-8aa6c9e4f941 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 769 time: 0.3732173 May 25 16:14:22.953190 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51600) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:22.958743 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'icmp', 'direction': 'egress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:22.965459 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:22.969751 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:22.972969 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:9. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:22.979170 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 9; quota limit: 100; headroom:91 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:23.079045 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.148410 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.211721 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.213803 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-7084ad7f-c6f4-49c9-bd4e-a9b4f8b82417 tempest-TestShelveInstance-1882661298 tempest-TestShelveInstance-1882661298-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:23.238110 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=b01172b5-a8aa-4c22-9ebd-6a72882384ee,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:23.240683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:23.606285 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.607481 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-a2a9761b-68f9-4a75-b7da-79765813c366 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 768 time: 0.6533380 May 25 16:14:23.610399 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51604) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:23.616379 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'ingress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:23.622935 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:23.627646 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:23.631023 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:10. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:23.637941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 10; quota limit: 100; headroom:90 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:23.765669 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.securitygroups_db [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Project 7413f95c7d284674bb0d587de6b42810 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:14:23.772435 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.842331 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.920245 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:23.922926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-c6ff1ccb-b7cc-47d7-9463-0ef2f747b8c6 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:23.945683 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=df2988ff-5ff2-4127-8302-76cd558503a1,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:23.950575 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:23.999766 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:24.001001 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-d08b257a-a116-43a2-bfde-8d8a6596cbed tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 774 time: 0.3891540 May 25 16:14:24.003857 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51608) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:14:24.008917 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'security_group_rule': {'security_group_id': '134145c7-9f57-4f74-a5b6-0e989035223a', 'project_id': '7413f95c7d284674bb0d587de6b42810', 'protocol': 'icmp', 'ethertype': 'IPv6', 'direction': 'egress'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:24.015518 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:14:24.019390 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Usage tracker for resource:security_group_rule and tenant:7413f95c7d284674bb0d587de6b42810 is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:14:24.022429 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Quota usage for security_group_rule was recalculated. Used quota:11. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:14:24.029966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attempting to reserve 1 items for resource security_group_rule. Total usage: 11; quota limit: 100; headroom:89 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:14:24.137869 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.securitygroups_db [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Project 7413f95c7d284674bb0d587de6b42810 added a security group rule with legacy IPv6 ICMP protocol name icmp, ipv6-icmp should be used instead. It was automatically converted. May 25 16:14:24.144377 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:24.214138 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:24.383176 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304798'] for security_group_rule, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:24.385278 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:24.412675 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'SecurityGroupRule': ['ID=83b29368-2eee-4e25-ae0f-0108142e8348,revision_number=0']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:24.419489 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.030s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:24.508378 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group_rule, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:24.509534 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e284884d-89d7-43b9-bee1-543369d8f03e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "POST /v2.0/security-group-rules HTTP/1.1" status: 201 len: 773 time: 0.5047233 May 25 16:14:24.635482 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-d02f5ff9-8f37-466c-8764-fb37269d756f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1667647037&fields=id&tenant_id=7413f95c7d284674bb0d587de6b42810 HTTP/1.1" status: 200 len: 267 time: 0.0439644 May 25 16:14:24.750116 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-f9dba07f-ad52-4d22-8eef-9f1b934ff58b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:24.751700 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-f9dba07f-ad52-4d22-8eef-9f1b934ff58b tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.1108482 May 25 16:14:24.764974 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'port': {'security_groups': ['8286425c-b378-48e3-9125-58c869823a8e', '134145c7-9f57-4f74-a5b6-0e989035223a']}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:14:24.893506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:25.168111 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:14:25.193983 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:25.342012 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:25.347832 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2d3a4091-ee16-451f-839b-f4cc8af38263 tempest-AttachVolumeShelveTestJSON-39517098 tempest-AttachVolumeShelveTestJSON-39517098-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:14:25.377292 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:14:25.422624 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:25.423336 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:25.425078 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-fa7caec6-5915-4eec-85ca-f9e14485b709 req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "PUT /v2.0/ports/145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 981 time: 0.6665697 May 25 16:14:25.425629 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:14:25.451429 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=6']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:14:25.454717 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.107s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:14:25.487257 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:14:25.567160 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:14:25.698123 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-3a25f0b0-e61b-4358-8edd-8b5d5fc6834f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:25.699916 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-3a25f0b0-e61b-4358-8edd-8b5d5fc6834f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1137 time: 0.1597855 May 25 16:14:25.712479 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3db77413-b425-4239-af37-b454f200ca1d req-b588ffdb-8722-44eb-9149-80341bedbd1f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:25.713997 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3db77413-b425-4239-af37-b454f200ca1d req-b588ffdb-8722-44eb-9149-80341bedbd1f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.1345060 May 25 16:14:25.821264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-cca63b44-e39a-4490-941e-891678c02119 None None] Provisioning complete for port 145301a2-46f6-4ded-b871-52227a79d41a triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:14:25.821644 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-cca63b44-e39a-4490-941e-891678c02119 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:25.845463 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3db77413-b425-4239-af37-b454f200ca1d req-1eae9406-aeea-4dbd-9b9a-17f39c628826 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.1117377 May 25 16:14:25.884272 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-a998e814-12c3-4300-b5bc-e981d4888890 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:25.885836 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-a998e814-12c3-4300-b5bc-e981d4888890 service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.1703544 May 25 16:14:26.105358 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-e9aea635-58fc-46b9-876b-a846e01190c2 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 1001 time: 0.2126310 May 25 16:14:26.174932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-01777282-ffa0-4cad-ab19-ec829b28797a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:26.176683 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-01777282-ffa0-4cad-ab19-ec829b28797a service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0648439 May 25 16:14:26.271932 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-1653d5e2-e9b2-439d-a80b-b28cf822aed2 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:26.273503 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-1653d5e2-e9b2-439d-a80b-b28cf822aed2 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0908675 May 25 16:14:26.436662 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-f66c9daf-d28b-49a2-a027-aaf7d4f76dcc service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1555195 May 25 16:14:26.579420 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1778f87a-1e22-44e7-933f-9cb58a806a27 req-4558c85c-a450-4c67-9997-d6b3a45c5342 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1358876 May 25 16:14:35.656712 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-bc17fe7e-ca3f-4d64-b801-143488531f4c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:35.786252 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b5c81a61-45fa-4a95-aded-559dbcfd3d71 req-9f3b8de5-79d7-411c-ae21-e0cd88b90513 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:35.787757 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5c81a61-45fa-4a95-aded-559dbcfd3d71 req-9f3b8de5-79d7-411c-ae21-e0cd88b90513 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0874894 May 25 16:14:35.883290 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b5c81a61-45fa-4a95-aded-559dbcfd3d71 req-64974997-8b78-4a73-8316-61fb621e4a2d tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0614040 May 25 16:14:35.901207 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-fa9283de-9c7a-486a-94f7-6801807b9953 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:35.901207 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-fa9283de-9c7a-486a-94f7-6801807b9953 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1137 time: 0.1119354 May 25 16:14:36.037824 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-170ff031-108c-4bb6-bb58-16387346b6ab service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:36.039301 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-170ff031-108c-4bb6-bb58-16387346b6ab service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.1283591 May 25 16:14:36.155469 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-c1f1f1fd-dbc8-413d-bf0b-e1bae2685675 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 1001 time: 0.1041441 May 25 16:14:36.217466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-5ecea1f5-610e-4035-94d7-d1bc1f0bfa70 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:36.218930 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-5ecea1f5-610e-4035-94d7-d1bc1f0bfa70 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0607994 May 25 16:14:36.301174 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-695b98b7-0411-443b-a8c0-4bf4d68f59ba service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:36.302668 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-695b98b7-0411-443b-a8c0-4bf4d68f59ba service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0771823 May 25 16:14:36.441736 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-f6a406ef-5835-44c7-8d31-0d90a2dd19a5 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1321480 May 25 16:14:36.576363 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-57620e6b-8b71-4e6b-8b0b-bf6fc43f2464 req-9499015f-61f0-4a35-bdce-7ae33a3e1c7e service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1277869 May 25 16:14:37.075002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7419254b-520b-42c4-908c-8200e22dab07 req-73d2b23e-d45a-4d39-b192-061880ea9780 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:37.077113 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7419254b-520b-42c4-908c-8200e22dab07 req-73d2b23e-d45a-4d39-b192-061880ea9780 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0810480 May 25 16:14:37.126887 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7419254b-520b-42c4-908c-8200e22dab07 req-dd15b905-d1e1-4852-bf96-4a4d0489be37 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0429020 May 25 16:14:38.170801 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:38.328954 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-a3e8cf1b-bbcc-41aa-bd25-cf004a67d153 req-173fd29a-4ca5-4c90-a2b1-1d3676e9c3d8 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:38.330706 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a3e8cf1b-bbcc-41aa-bd25-cf004a67d153 req-173fd29a-4ca5-4c90-a2b1-1d3676e9c3d8 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0908179 May 25 16:14:38.380852 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-a3e8cf1b-bbcc-41aa-bd25-cf004a67d153 req-8ad9bc26-161b-4091-9d65-aa5f26eb9da9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0439935 May 25 16:14:38.427975 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b419baca-4bc3-4ab7-a6b8-4b5867a05906 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:39.108198 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:39.302402 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:14:39.571745 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-743ed3be-2119-4800-9047-8444176fbc8f req-ac01a173-d17d-4ec1-81fd-eb5ff9997eba tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:39.573191 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-743ed3be-2119-4800-9047-8444176fbc8f req-ac01a173-d17d-4ec1-81fd-eb5ff9997eba tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0791345 May 25 16:14:39.619997 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-743ed3be-2119-4800-9047-8444176fbc8f req-d293e6de-b7b2-42e7-89e3-da666ed68354 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0405357 May 25 16:14:40.548545 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:40.906945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-7cf570ab-09af-4809-9841-7feb41033e7e req-6db7b964-6495-47a8-9aaf-4503b593efc6 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:40.908482 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7cf570ab-09af-4809-9841-7feb41033e7e req-6db7b964-6495-47a8-9aaf-4503b593efc6 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.1845603 May 25 16:14:40.957965 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-7cf570ab-09af-4809-9841-7feb41033e7e req-55fa5ce8-d2f2-4906-a4d9-4fceaf511432 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0434532 May 25 16:14:41.130582 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:14:42.152990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-bb14a59f-f43c-4dca-8568-aeec710fb1c1 req-64fe068a-5c2a-499a-9afb-d73e40e0a0c9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:42.154440 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bb14a59f-f43c-4dca-8568-aeec710fb1c1 req-64fe068a-5c2a-499a-9afb-d73e40e0a0c9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0858364 May 25 16:14:42.203563 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-bb14a59f-f43c-4dca-8568-aeec710fb1c1 req-8b899344-10b0-4ea0-a2ed-b51ef2dc549e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0426030 May 25 16:14:43.392575 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-9a13d5d5-6cf2-4328-a294-f149f888e4c4 req-314ac071-c9ed-4f53-9eec-2ad0e5824fd9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:43.394138 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9a13d5d5-6cf2-4328-a294-f149f888e4c4 req-314ac071-c9ed-4f53-9eec-2ad0e5824fd9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0816090 May 25 16:14:43.441926 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-9a13d5d5-6cf2-4328-a294-f149f888e4c4 req-1163abbf-88a2-4049-8665-3cf8f8f3dd2f tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0418549 May 25 16:14:44.631913 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-665d9709-f1a7-4746-92f3-73baa320c3ba req-f1d46e0e-287a-4c38-856b-eb7069e919e1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:44.633367 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-665d9709-f1a7-4746-92f3-73baa320c3ba req-f1d46e0e-287a-4c38-856b-eb7069e919e1 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0848870 May 25 16:14:44.680074 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-665d9709-f1a7-4746-92f3-73baa320c3ba req-5312b307-d923-482c-bd3c-66ba527e8f89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0409729 May 25 16:14:45.883952 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-c959f7fd-537a-4648-9ab6-35849aa26b07 req-4b44b7fd-512b-4572-b999-f4e1766acf8e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:45.885469 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c959f7fd-537a-4648-9ab6-35849aa26b07 req-4b44b7fd-512b-4572-b999-f4e1766acf8e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0825608 May 25 16:14:45.936091 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c959f7fd-537a-4648-9ab6-35849aa26b07 req-e8dcd14f-a6f1-4564-9898-8d7ca3e2aa17 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0448692 May 25 16:14:47.132686 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0a24c263-2cb9-4bbd-8bac-e1faeb0ff0e0 req-d8299179-b71c-47d4-bc6f-6e46db84ace2 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:47.134478 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0a24c263-2cb9-4bbd-8bac-e1faeb0ff0e0 req-d8299179-b71c-47d4-bc6f-6e46db84ace2 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0910783 May 25 16:14:47.190899 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0a24c263-2cb9-4bbd-8bac-e1faeb0ff0e0 req-6ab6e5d8-aeb9-40ab-b915-22e5a82041f3 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0469322 May 25 16:14:48.375042 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2123aa9c-34e3-4730-8552-594f86ee0dab req-261bc41c-fea5-482c-bb18-23a78afa7852 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:14:48.376525 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2123aa9c-34e3-4730-8552-594f86ee0dab req-261bc41c-fea5-482c-bb18-23a78afa7852 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.0814555 May 25 16:14:48.428359 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2123aa9c-34e3-4730-8552-594f86ee0dab req-89076629-cbe8-48a6-9ee3-281c2a449e97 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&id=134145c7-9f57-4f74-a5b6-0e989035223a&fields=id&fields=name HTTP/1.1" status: 200 len: 380 time: 0.0459881 May 25 16:15:05.650123 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e98e96c0-d341-439e-a3c7-f35c7fda31d7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:08.141230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:08.427136 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a9394b66-df86-40f6-9917-731fab1ea46b None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:09.141486 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:10.372575 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-de4cf388-da7d-471c-850b-ae972e2b980a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-821231136&fields=id&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 267 time: 0.0383360 May 25 16:15:10.522788 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6236f77c-7eb6-4e05-8f6f-80e42c328bca tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:10.524176 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6236f77c-7eb6-4e05-8f6f-80e42c328bca tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.1445675 May 25 16:15:10.539294 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-275c0334-aa9e-44ed-a043-0cfec094627d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/quotas/a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 375 time: 0.0093379 May 25 16:15:10.558535 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:10.629394 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-b5192c69-8754-49d7-8c86-15ffec6e747b tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&fields=id HTTP/1.1" status: 200 len: 440 time: 0.0847814 May 25 16:15:10.772173 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-922d29c1-bbf6-4c4b-9e63-3615b18189f8 service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1313431 May 25 16:15:10.907163 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-d9b7ef95-e756-4cea-a5eb-bcdd696ac5bc service neutron] 10.222.0.82 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1287744 May 25 16:15:11.147224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:11.646663 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-16b9e8df-944b-4595-a903-ed36cbeeec91 req-febdf630-d4e8-48ea-ab2f-3d82f9eebb23 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 210 time: 0.0477986 May 25 16:15:12.799949 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0d22b72-b002-4791-b9d7-f927392999e5 req-3c6a0101-2ead-4f64-af49-9124e3d849e2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 210 time: 0.0450256 May 25 16:15:13.971067 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ee2f4d1d-9cdb-4b67-bc81-38f56571f3c8 req-62a9eb10-8f7d-4da2-a2f5-280ee06724d3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 210 time: 0.0556417 May 25 16:15:13.985303 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6ce8c1d9-0605-4245-8948-b62641caec05 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent', 'provider:network_type', 'provider:physical_network', 'provider:segmentation_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:13.986734 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6ce8c1d9-0605-4245-8948-b62641caec05 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 811 time: 0.1968222 May 25 16:15:14.046149 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-066969ab-2677-4a78-8552-81f78bf02cfa tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/security-groups?fields=id&fields=name&tenant_id=a1e31a264b7949bea6aa47cb7781106b HTTP/1.1" status: 200 len: 379 time: 0.0439312 May 25 16:15:14.054829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'port': {'device_id': '58e18ce7-8323-44d8-aa49-6157273d7138', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'admin_state_up': True, 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'security_groups': ['dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64']}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:15:14.299561 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:15:14.310958 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.quota.driver [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource port. Total usage: 4; quota limit: 500; headroom:496 {{(pid=99545) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:15:14.363894 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Made reservation on behalf of a1e31a264b7949bea6aa47cb7781106b for: {'port': 1} {{(pid=99545) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}} May 25 16:15:14.367642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:14.534403 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_common [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.20.0.4 (6c61cff6-814b-4af0-8599-e026b705cc35/a4d8980e-d38c-4165-af50-ed2f39dc708c/c3ce7b44-762a-4e22-a20f-491edca650be) {{(pid=99545) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:15:14.725291 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:14.740760 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Transition to ACTIVE for port object c3ce7b44-762a-4e22-a20f-491edca650be will not be triggered until provisioned by entity DHCP. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:15:14.855783 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_create {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:14.861741 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-c2edfd5e-5352-4c8c-9291-f381ffc99346 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:14.930830 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.dhcp_agent_scheduler [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 6c61cff6-814b-4af0-8599-e026b705cc35 is already hosted by enough agents. {{(pid=99545) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:15:14.933981 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.base_scheduler [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99545) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:15:14.938077 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99545) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:15:14.970362 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-6de27118-eef3-4677-9566-1d71f97749f4 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0828650 May 25 16:15:14.975491 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=1']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:14.982731 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.121s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:15.016344 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details', 'network'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.016968 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:15.083150 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-84afef1f-feaf-4a8c-b982-e1c083d24a28 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.083472 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-84afef1f-feaf-4a8c-b982-e1c083d24a28 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1137 time: 0.0789788 May 25 16:15:15.134719 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "POST /v2.0/ports HTTP/1.1" status: 201 len: 933 time: 1.0835640 May 25 16:15:15.153618 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Request body: {'port': {'device_id': '58e18ce7-8323-44d8-aa49-6157273d7138', 'device_owner': 'compute:nova', 'binding:host_id': 'hv-cinder-78638'}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:15:15.235493 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-082d180c-c208-4ca7-95d6-8a4a3e5e2ade service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.237335 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-082d180c-c208-4ca7-95d6-8a4a3e5e2ade service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.1346078 May 25 16:15:15.292493 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e879870d-d3a7-40ce-a552-74472e0edbac req-26e4e66e-3fc6-4f8f-86ca-80bf8d2a9d0e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.293276 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e879870d-d3a7-40ce-a552-74472e0edbac req-26e4e66e-3fc6-4f8f-86ca-80bf8d2a9d0e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:15:15.297342 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:15.312668 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-56891715-02cb-4201-a141-499d71517e77 None None] Provisioning for port c3ce7b44-762a-4e22-a20f-491edca650be completed by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:15:15.320012 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-56891715-02cb-4201-a141-499d71517e77 None None] Provisioning complete for port c3ce7b44-762a-4e22-a20f-491edca650be triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:15:15.320681 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-56891715-02cb-4201-a141-499d71517e77 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:15.346462 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-475a1bb8-5705-4d69-85e3-3bb24582d112 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 1001 time: 0.1027751 May 25 16:15:15.398253 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-56891715-02cb-4201-a141-499d71517e77 None None] Port c3ce7b44-762a-4e22-a20f-491edca650be cannot update to ACTIVE because it is not bound. {{(pid=99546) _port_provisioned /opt/stack/neutron/neutron/plugins/ml2/plugin.py:337}} May 25 16:15:15.424002 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-cd902d7b-e882-46ab-8018-63f17c4ce8b3 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.426075 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-cd902d7b-e882-46ab-8018-63f17c4ce8b3 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0738437 May 25 16:15:15.433552 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-e879870d-d3a7-40ce-a552-74472e0edbac req-26e4e66e-3fc6-4f8f-86ca-80bf8d2a9d0e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:15:15.434712 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e879870d-d3a7-40ce-a552-74472e0edbac req-26e4e66e-3fc6-4f8f-86ca-80bf8d2a9d0e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 931 time: 0.3488185 May 25 16:15:15.503824 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-dd95487d-12a6-428d-a41a-8d774e4209ba service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:15.505196 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-dd95487d-12a6-428d-a41a-8d774e4209ba service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0728838 May 25 16:15:15.567579 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e879870d-d3a7-40ce-a552-74472e0edbac req-22c1231a-332c-40cd-a27a-73d77563e3ff tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1193714 May 25 16:15:15.727022 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Ignoring state change previous_port_status: DOWN current_port_status: DOWN port_id c3ce7b44-762a-4e22-a20f-491edca650be {{(pid=99545) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:15:15.735251 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:15.753661 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.provisioning_blocks [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Transition to ACTIVE for port object c3ce7b44-762a-4e22-a20f-491edca650be will not be triggered until provisioned by entity L2. {{(pid=99545) add_provisioning_component /opt/stack/neutron/neutron/db/provisioning_blocks.py:73}} May 25 16:15:15.765967 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-bbcc6614-f3f1-471c-bdc1-5ff1aba57535 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2516360 May 25 16:15:15.815976 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:15.820130 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0a73e7ae-d314-47ac-a8d5-7ac13c591bb1 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:15.911419 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [req-e2d73c34-f8ed-4b1b-9093-ac204796cecb req-04a1a47c-83d7-43e6-8c24-6a131299c14d service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1408050 May 25 16:15:15.981991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Pushing event updated for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=2']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:15.990074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.170s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:16.068829 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Attempting to bind port c3ce7b44-762a-4e22-a20f-491edca650be on host hv-cinder-78638 for vnic_type normal with profile {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/managers.py:790}} May 25 16:15:16.069139 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Attempting to bind port c3ce7b44-762a-4e22-a20f-491edca650be by drivers openvswitch,hyperv on host hv-cinder-78638 at level 0 using segments [{'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:815}} May 25 16:15:16.069461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Attempting to bind port c3ce7b44-762a-4e22-a20f-491edca650be on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:15:16.074179 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Port c3ce7b44-762a-4e22-a20f-491edca650be on network 6c61cff6-814b-4af0-8599-e026b705cc35 not bound, no agent of type Open vSwitch agent registered on host hv-cinder-78638 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:98}} May 25 16:15:16.074486 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Attempting to bind port c3ce7b44-762a-4e22-a20f-491edca650be on network 6c61cff6-814b-4af0-8599-e026b705cc35 {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:86}} May 25 16:15:16.080358 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Checking agent: {'id': '5e685292-24fb-4d64-9f07-4aea0feb7f0e', 'agent_type': 'HyperV agent', 'binary': 'neutron-hyperv-agent', 'topic': 'N/A', 'host': 'hv-cinder-78638', 'admin_state_up': True, 'created_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'started_at': datetime.datetime(2021, 5, 25, 16, 1, 9), 'heartbeat_timestamp': datetime.datetime(2021, 5, 25, 16, 15, 10), 'description': None, 'resources_synced': None, 'availability_zone': None, 'alive': True, 'configurations': {'vswitch_mappings': {'physnet1$': 'br-data'}}, 'resource_versions': {}} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:105}} May 25 16:15:16.080698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Checking segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} for mappings: {'physnet1$': 'br-data'} with network types: ['local', 'flat', 'vlan'] {{(pid=99545) check_segment_for_agent /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:341}} May 25 16:15:16.081211 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.mech_agent [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Bound using segment: {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'} {{(pid=99545) bind_port /opt/stack/neutron/neutron/plugins/ml2/drivers/mech_agent.py:115}} May 25 16:15:16.081641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.managers [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Bound port: c3ce7b44-762a-4e22-a20f-491edca650be, host: hv-cinder-78638, vif_type: hyperv, vif_details: {"connectivity": "legacy", "port_filter": false}, binding_levels: [{'bound_driver': 'hyperv', 'bound_segment': {'id': 'bd457928-332b-49b5-8448-df2758dfb4c0', 'network_type': 'vlan', 'physical_network': 'physnet1', 'segmentation_id': 902, 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35'}}] {{(pid=99545) _bind_port_level /opt/stack/neutron/neutron/plugins/ml2/managers.py:928}} May 25 16:15:16.081990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:16.173996 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, cleared binding levels {{(pid=99545) clear_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:88}} May 25 16:15:16.177877 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, set binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=c3ce7b44-762a-4e22-a20f-491edca650be,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) set_binding_levels /opt/stack/neutron/neutron/plugins/ml2/db.py:60}} May 25 16:15:16.244482 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:16.248517 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-e3a924aa-4f7f-45b8-867b-f81f5c7b2404 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:16.301781 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:15:16.310533 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:16.311046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:16.312829 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] 10.222.0.42 "PUT /v2.0/ports/c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 200 len: 1093 time: 1.1635299 May 25 16:15:16.313203 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:16.340326 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Pushing event updated for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=3']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:16.343823 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.095s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:16.515215 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-063e7e9d-51f1-416a-977c-c31290c45f85 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:16.516676 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-063e7e9d-51f1-416a-977c-c31290c45f85 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1096 time: 0.0835915 May 25 16:15:16.581533 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'status': 'completed', 'code': 200} May 25 16:15:16.708142 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-e09640fe-9507-4303-8459-b9327794651d service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 200 len: 217 time: 0.1773503 May 25 16:15:16.831991 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-1de049a8-4220-400f-9169-6ed5ebffad59 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:16.833378 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-1de049a8-4220-400f-9169-6ed5ebffad59 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0823944 May 25 16:15:16.892364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-479e2531-7fd6-434a-98c8-aab8047c00f4 req-7fb3d463-8d51-43a1-acbe-ac303acb7129 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:16.894464 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-479e2531-7fd6-434a-98c8-aab8047c00f4 req-7fb3d463-8d51-43a1-acbe-ac303acb7129 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.1888955 May 25 16:15:16.952430 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-479e2531-7fd6-434a-98c8-aab8047c00f4 req-952f88e4-f795-40fc-b0dc-d539b946478d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0519290 May 25 16:15:16.963482 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6c67729f-f1aa-4561-ab07-215e95b841ca service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:16.964983 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-6c67729f-f1aa-4561-ab07-215e95b841ca service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1221559 May 25 16:15:17.107716 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-c96e5fa4-68cc-47c4-b528-7de10a67a9ac service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1348906 May 25 16:15:17.244248 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-eb4805e3-dc76-4ea2-a36b-a4951e23719c req-1fb12f76-d859-4dd3-a331-8e9e218d0f69 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1303401 May 25 16:15:17.464673 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75571011-db86-4b2c-8204-96b699eb42b8 req-8c00c181-1fb4-493a-a676-6811132dadb8 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:17.466135 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-8c00c181-1fb4-493a-a676-6811132dadb8 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1096 time: 0.0771577 May 25 16:15:17.604199 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75571011-db86-4b2c-8204-96b699eb42b8 req-7e3238ae-6476-48f6-9101-13a9b8bd5f7b service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:17.605643 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-7e3238ae-6476-48f6-9101-13a9b8bd5f7b service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1325886 May 25 16:15:17.673944 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-94d8c62c-d8af-448f-9529-e5b41a490504 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 200 len: 217 time: 0.0606925 May 25 16:15:17.744831 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75571011-db86-4b2c-8204-96b699eb42b8 req-c3ae77c6-0952-4caf-8243-58cc945112cd service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:17.746172 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-c3ae77c6-0952-4caf-8243-58cc945112cd service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0647113 May 25 16:15:17.829812 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-75571011-db86-4b2c-8204-96b699eb42b8 req-bc319240-3738-4737-99ec-e8266e463f17 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:17.831942 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-bc319240-3738-4737-99ec-e8266e463f17 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0761433 May 25 16:15:17.983867 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-db69aad6-bccc-4f6e-8625-6f4d4cebb909 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1426475 May 25 16:15:18.198642 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-75571011-db86-4b2c-8204-96b699eb42b8 req-b0de7007-dd26-4666-b5b4-cd66ef226c50 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.2078671 May 25 16:15:18.218114 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-e96f0380-0330-409e-b154-5c23edb7434c req-ad70c959-1e34-4d8b-a221-a21e242c773c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:18.219784 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e96f0380-0330-409e-b154-5c23edb7434c req-ad70c959-1e34-4d8b-a221-a21e242c773c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.1459041 May 25 16:15:18.271412 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-e96f0380-0330-409e-b154-5c23edb7434c req-140879d6-5cb8-4a83-95f3-42c8cb7b2599 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0424049 May 25 16:15:19.459850 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4a9756d4-c797-442f-a18f-2a2c05a7bb85 req-c0743333-d4ad-467c-9365-b6692d2fad7e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:19.461228 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a9756d4-c797-442f-a18f-2a2c05a7bb85 req-c0743333-d4ad-467c-9365-b6692d2fad7e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.0816810 May 25 16:15:19.507383 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4a9756d4-c797-442f-a18f-2a2c05a7bb85 req-3877e82b-bbd1-4bcd-b45d-50b64dedd4e1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0387781 May 25 16:15:20.700224 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2f0f3eca-3e78-4adf-af2d-6b143476b1bf req-3296d21d-191b-4657-b226-8ebbc9a7bab1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:20.701738 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2f0f3eca-3e78-4adf-af2d-6b143476b1bf req-3296d21d-191b-4657-b226-8ebbc9a7bab1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.0845737 May 25 16:15:20.748719 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2f0f3eca-3e78-4adf-af2d-6b143476b1bf req-216cdc8e-0321-47e1-9815-5dc64e6f33e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0402756 May 25 16:15:21.604858 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caaf62e3-582b-4403-b223-21d51dbba156 req-c26595b4-523e-42cb-8c7f-5ceb91495868 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?name=tempest-secgroup-smoke-1667647037&fields=id&tenant_id=7413f95c7d284674bb0d587de6b42810 HTTP/1.1" status: 200 len: 267 time: 0.0418191 May 25 16:15:21.800143 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caaf62e3-582b-4403-b223-21d51dbba156 req-90b584fb-0406-42d5-8ff9-f3a964f010ef tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:21.801622 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caaf62e3-582b-4403-b223-21d51dbba156 req-90b584fb-0406-42d5-8ff9-f3a964f010ef tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 984 time: 0.1911902 May 25 16:15:21.812339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.v2.base [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Request body: {'port': {'security_groups': ['8286425c-b378-48e3-9125-58c869823a8e']}} {{(pid=99545) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:15:21.986772 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.038284 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-153eb150-dcca-4217-8291-8f7732b04934 req-f09e9418-4a74-48e7-8bf5-a32c87f8e01a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:22.039665 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-153eb150-dcca-4217-8291-8f7732b04934 req-f09e9418-4a74-48e7-8bf5-a32c87f8e01a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.1810005 May 25 16:15:22.122136 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-153eb150-dcca-4217-8291-8f7732b04934 req-e372a65c-7677-48f0-9265-78ee2585e546 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0767705 May 25 16:15:22.291881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:22.319814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for port, precommit_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.427144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-1923045'] for port, after_update {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.433345 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9e449a24-acf6-4dd7-b141-15123c51e669 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:22.466707 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:15:22.499948 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:22.500598 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.502051 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-caaf62e3-582b-4403-b223-21d51dbba156 req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "PUT /v2.0/ports/145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 942 time: 0.6942823 May 25 16:15:22.503172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:22.510269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51740) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:22.538941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event updated for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=7']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:22.557917 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:15:22.559652 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.560226 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.560773 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.561306 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.561866 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.562401 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.562964 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.563488 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.564026 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.564558 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.565197 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.565732 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.575530 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.142s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:22.607136 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.641473 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.663673 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:15:22.695052 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-cc78e9b7-7d68-4ce0-9905-ccf38c38bd70 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:22.704937 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-cc78e9b7-7d68-4ce0-9905-ccf38c38bd70 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1098 time: 0.1033325 May 25 16:15:22.740633 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-8ee5f590-0c01-4663-8006-f8f7e9fee5c3 None None] Provisioning complete for port 145301a2-46f6-4ded-b871-52227a79d41a triggered by entity DHCP. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:15:22.741004 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8ee5f590-0c01-4663-8006-f8f7e9fee5c3 None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.804243 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.809324 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d3b987c4-8a2a-42ea-ac42-e915da5e8ed6 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:22.825128 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=134145c7-9f57-4f74-a5b6-0e989035223a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:22.836479 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.836861 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.837232 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.837601 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.837966 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.838324 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.838679 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.842028 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.842478 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.842908 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.843276 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.843666 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:22.847973 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:22.849229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:22.850367 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-02c1820b-0b9e-4dc5-89ab-28d39d6ffe89 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "DELETE /v2.0/security-groups/134145c7-9f57-4f74-a5b6-0e989035223a HTTP/1.1" status: 204 len: 168 time: 0.3383338 May 25 16:15:22.855219 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51744) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:22.874596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-75ed424f-373b-45d3-afc8-20fd98cf7dc6 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:22.877745 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-75ed424f-373b-45d3-afc8-20fd98cf7dc6 service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.1643589 May 25 16:15:22.995936 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-5efd46fa-b94d-4acd-9cbc-e7bd52cfef91 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 1001 time: 0.1110265 May 25 16:15:23.061122 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-090b0421-e432-45bf-8c5c-52d4774bbc32 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:23.062657 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-090b0421-e432-45bf-8c5c-52d4774bbc32 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0595443 May 25 16:15:23.140614 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Deleting port df27303f-a70f-49d6-8356-b5f8d0e19eaa {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:23.141085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:23.141485 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-325c1ef7-902d-4bf1-9c70-fc92f4ceef72 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:23.143538 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-325c1ef7-902d-4bf1-9c70-fc92f4ceef72 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0733945 May 25 16:15:23.293993 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:23.294423 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:23.361537 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-2922163d-7bcc-4aab-8d50-09dc0edae2b0 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.2068226 May 25 16:15:23.394135 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b132e108-8f54-4746-8999-29fd2ebdd343 req-e6836a56-15bd-4bf6-a9d9-4dc570447110 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:23.395653 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b132e108-8f54-4746-8999-29fd2ebdd343 req-e6836a56-15bd-4bf6-a9d9-4dc570447110 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.1528709 May 25 16:15:23.426696 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Calling delete_port for df27303f-a70f-49d6-8356-b5f8d0e19eaa owned by network:floatingip {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:23.524745 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b132e108-8f54-4746-8999-29fd2ebdd343 req-a7a21bba-2cb1-4660-b6d4-05062687766c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0743120 May 25 16:15:23.560765 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-dbdb770e-635f-472e-bc2f-1aa4a5795655 req-d18475de-457c-4447-bf58-cfed4d944b99 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1914489 May 25 16:15:23.638580 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:23.642902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-eda0b3d4-f559-46b4-815a-c4c9a718b65d service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:23.691035 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event deleted for resources: {'Port': ['ID=df27303f-a70f-49d6-8356-b5f8d0e19eaa,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:23.724008 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.081s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:23.788433 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks [] for floatingip, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:23.788990 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Floating IP c4abf34e-d7c2-4615-a16a-a8c322bada22 disassociated (deleted). External IP: 172.27.2.185, port: 145301a2-46f6-4ded-b871-52227a79d41a. May 25 16:15:24.263486 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Router 0492512e-ccab-4c81-9788-1e5b615f9dc3 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:15:24.452777 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:15:24.464259 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Sync routers for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:15:24.475173 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:24.479151 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:24.541631 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device c3ce7b44-762a-4e22-a20f-491edca650be details requested by agent hyperv_hv-cinder-78638 with host hv-cinder-78638 {{(pid=99546) get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:88}} May 25 16:15:24.565838 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:24.631122 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "DELETE /v2.0/floatingips/c4abf34e-d7c2-4615-a16a-a8c322bada22 HTTP/1.1" status: 204 len: 168 time: 1.7700717 May 25 16:15:24.631122 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:24.748863 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:24.788068 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-10a3cd21-23cf-41c9-b318-c77f40be3077 req-7237c185-f773-449f-9abc-9cc6dc3f65c3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:24.789601 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10a3cd21-23cf-41c9-b318-c77f40be3077 req-7237c185-f773-449f-9abc-9cc6dc3f65c3 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 943 time: 0.1401238 May 25 16:15:24.843993 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-10a3cd21-23cf-41c9-b318-c77f40be3077 req-5b97c951-b94a-44c2-a1e1-05af02ae7059 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0474768 May 25 16:15:24.935722 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=c3ce7b44-762a-4e22-a20f-491edca650be,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:24.997204 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:15:25.007404 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Returning: {'device': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'port_id': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'mac_address': 'fa:16:3e:64:d4:d5', 'admin_state_up': True, 'network_type': 'vlan', 'segmentation_id': 902, 'physical_network': 'physnet1', 'mtu': 1500, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.4'}], 'device_owner': 'compute:nova', 'allowed_address_pairs': [], 'port_security_enabled': True, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False} {{(pid=99546) _get_device_details /opt/stack/neutron/neutron/plugins/ml2/rpc.py:168}} May 25 16:15:25.104369 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:25.114019 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-8445cc27-2b43-4524-a110-70eed1467ca0 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:25.119977 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-8445cc27-2b43-4524-a110-70eed1467ca0 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=7413f95c7d284674bb0d587de6b42810&device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 1098 time: 0.0805318 May 25 16:15:25.155729 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Ignoring state change previous_port_status: DOWN current_port_status: BUILD port_id c3ce7b44-762a-4e22-a20f-491edca650be {{(pid=99546) record_port_status_changed /opt/stack/neutron/neutron/notifiers/nova.py:224}} May 25 16:15:25.187110 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=c3ce7b44-762a-4e22-a20f-491edca650be,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:25.215954 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:25.220371 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Checking router: 0492512e-ccab-4c81-9788-1e5b615f9dc3 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:15:25.231426 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:25.231661 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:25.297169 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:25.308351 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-66341aab-4f3a-4b08-ac75-89234f443250 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:25.405394 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-af3a0a6d-eb62-4a86-b104-2521efe15711 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:25.407089 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-af3a0a6d-eb62-4a86-b104-2521efe15711 service neutron] 10.222.0.42 "GET /v2.0/networks?id=ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 200 len: 916 time: 0.2785153 May 25 16:15:25.486072 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-052bfb13-ee01-41db-9dcd-516ff8de5823 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 200 len: 217 time: 0.0722313 May 25 16:15:25.510643 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=4']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:25.524280 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.524551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.524891 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.525232 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.525571 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.525897 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.526221 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.526551 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.526898 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.527225 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.527548 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.527881 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:15:25.536033 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.228s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:25.549848 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:25.560760 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-484e807b-b92e-4189-ae2a-53871f74e61d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:25.562383 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-484e807b-b92e-4189-ae2a-53871f74e61d service neutron] 10.222.0.42 "GET /v2.0/subnets?id=70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 200 len: 854 time: 0.0698900 May 25 16:15:25.616845 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['c3ce7b44-762a-4e22-a20f-491edca650be'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:15:25.650169 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-629d5f9f-3429-4270-9515-dcca7d39201f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:25.651709 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-629d5f9f-3429-4270-9515-dcca7d39201f service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=ceb9e841-143d-468e-800d-756d578d900d&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0815318 May 25 16:15:25.800635 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-ff53755f-2836-4f3d-bf3f-2f6e54dd318c service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1385469 May 25 16:15:25.948885 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ddf7073e-b672-4f72-ad7a-fb8958156e8d req-c328ff63-90b9-48c0-ad46-6ee477d0e2e4 service neutron] 10.222.0.42 "GET /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1455872 May 25 16:15:26.043149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-b0b9cc51-c3e7-483c-b8da-bf8d35fcd01a req-ce555f26-72d0-47f9-add2-552dd80b7a4c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:26.044674 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0b9cc51-c3e7-483c-b8da-bf8d35fcd01a req-ce555f26-72d0-47f9-add2-552dd80b7a4c tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 944 time: 0.0841556 May 25 16:15:26.090105 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-b0b9cc51-c3e7-483c-b8da-bf8d35fcd01a req-2b10a09c-ad5f-4fb1-bb62-89c6cfde2a36 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0392342 May 25 16:15:26.091157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:26.106022 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-57d30d88-3979-4af6-8b81-d8af3aa9833d None None] The sync data for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi is: [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:13:09Z', 'revision_number': 4, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '61c3265e-fd7c-491c-8373-f3de5eca4718', 'name': '', 'network_id': 'ceb9e841-143d-468e-800d-756d578d900d', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'mac_address': 'fa:16:3e:98:f6:4d', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_interface', 'standard_attr_id': 658, 'fixed_ips': [{'subnet_id': '70734104-38bb-429b-abc3-63fbc11e4122', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:08Z', 'updated_at': '2021-05-25T16:13:16Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'subnets': [{'id': '70734104-38bb-429b-abc3-63fbc11e4122', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:15:26.371897 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-c3c99d2f-8a93-4cbb-9c36-780a308eefbe None None] New status for floating IP c4abf34e-d7c2-4615-a16a-a8c322bada22: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:15:27.288941 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4431fa48-b9ea-4683-802a-3a5813513d88 req-e12fd33e-a152-4242-bc33-96c53db1b8fc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:27.290526 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4431fa48-b9ea-4683-802a-3a5813513d88 req-e12fd33e-a152-4242-bc33-96c53db1b8fc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 944 time: 0.0883813 May 25 16:15:27.338262 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4431fa48-b9ea-4683-802a-3a5813513d88 req-55ccee82-0bb1-48d5-a8d1-c2a64e6322f2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0411561 May 25 16:15:28.528596 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-065e6794-3784-4e7e-b531-3f91234dbc63 req-90de0b2a-8146-4804-9c57-fbf809ad28ad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:28.530070 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-065e6794-3784-4e7e-b531-3f91234dbc63 req-90de0b2a-8146-4804-9c57-fbf809ad28ad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 944 time: 0.0829291 May 25 16:15:28.548994 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device c3ce7b44-762a-4e22-a20f-491edca650be up at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_up /opt/stack/neutron/neutron/plugins/ml2/rpc.py:278}} May 25 16:15:28.582497 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-065e6794-3784-4e7e-b531-3f91234dbc63 req-f792ccf3-13ff-4dde-82fa-8da60788d26e tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0458782 May 25 16:15:28.735174 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning for port c3ce7b44-762a-4e22-a20f-491edca650be completed by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:133}} May 25 16:15:28.740827 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.provisioning_blocks [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Provisioning complete for port c3ce7b44-762a-4e22-a20f-491edca650be triggered by entity L2. {{(pid=99546) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}} May 25 16:15:28.743400 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-16161575'] for port, provisioning_complete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:28.868931 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:28.903264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be'}] {{(pid=99546) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:28.913612 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=c3ce7b44-762a-4e22-a20f-491edca650be,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:28.952389 dv-cinder-786384-6-iscsi neutron-server[99546]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'name': 'network-vif-plugged', 'status': 'completed', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'code': 200} May 25 16:15:29.021269 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966'] for port, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:29.023554 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:29.094967 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Pushing event updated for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=5']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:29.098667 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.075s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:29.423543 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-74bd0bd6-2fe8-4434-91c5-b982c808137e req-c9502ed4-36c7-49b3-8c10-3e425e1b2d15 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:29.425056 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74bd0bd6-2fe8-4434-91c5-b982c808137e req-c9502ed4-36c7-49b3-8c10-3e425e1b2d15 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1098 time: 0.0728016 May 25 16:15:29.472988 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-74bd0bd6-2fe8-4434-91c5-b982c808137e req-8d600bbb-f4bb-4ada-bdbb-1f87a58954f3 service neutron] 10.222.0.42 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0419505 May 25 16:15:29.830850 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-25e8bb28-7be4-49a4-bc31-f3276a2c86b3 req-1adf825d-3343-4a40-aecc-f5756548f275 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:29.832244 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25e8bb28-7be4-49a4-bc31-f3276a2c86b3 req-1adf825d-3343-4a40-aecc-f5756548f275 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.1440327 May 25 16:15:29.839747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-aeafc649-d0b6-4e70-a8de-796cd768cffe req-7ae0f44d-2e2a-4b4e-b241-704895493760 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:29.842287 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aeafc649-d0b6-4e70-a8de-796cd768cffe req-7ae0f44d-2e2a-4b4e-b241-704895493760 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.1219285 May 25 16:15:29.914275 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-25e8bb28-7be4-49a4-bc31-f3276a2c86b3 req-ad5ab954-9237-4ebe-8075-76d40d005424 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0722146 May 25 16:15:29.918643 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-aeafc649-d0b6-4e70-a8de-796cd768cffe req-285cffb5-dd90-4a68-9248-8f965a68e127 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0638161 May 25 16:15:31.169173 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-acfcd3da-2257-43a5-ba10-fabc1bfb544b req-6cd31829-21e9-4b21-9e6c-50483f97128e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:31.170620 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-acfcd3da-2257-43a5-ba10-fabc1bfb544b req-6cd31829-21e9-4b21-9e6c-50483f97128e tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.1420805 May 25 16:15:31.178754 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-3c9157f3-5148-4d30-aadf-533e64da0a26 req-360860b6-a6b3-497c-971d-309816e94105 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:31.180199 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c9157f3-5148-4d30-aadf-533e64da0a26 req-360860b6-a6b3-497c-971d-309816e94105 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.1534927 May 25 16:15:31.247788 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-acfcd3da-2257-43a5-ba10-fabc1bfb544b req-bf6e4ade-1908-4856-a496-64cb6f81e157 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0664911 May 25 16:15:31.255966 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-3c9157f3-5148-4d30-aadf-533e64da0a26 req-bef63f0e-1dbe-48cc-bc5b-bfcd34e1f093 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0613241 May 25 16:15:31.444049 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-149ac5f0-0117-4d2b-95f9-fac31cf29ee0 req-520d5509-1089-4d16-bcfb-ed927d0fcaa8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:31.445468 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-149ac5f0-0117-4d2b-95f9-fac31cf29ee0 req-520d5509-1089-4d16-bcfb-ed927d0fcaa8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.0798790 May 25 16:15:31.499573 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-149ac5f0-0117-4d2b-95f9-fac31cf29ee0 req-df3196e2-d046-46ef-9cad-c22d57e50913 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0480311 May 25 16:15:31.523567 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51798) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:31.607991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [None req-764e6850-ecd9-4f61-949e-cacacfc639d0 tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99544) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:31.609505 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-764e6850-ecd9-4f61-949e-cacacfc639d0 tempest-TestVolumeBootPattern-484335338 tempest-TestVolumeBootPattern-484335338-project-admin] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1093 time: 0.0848389 May 25 16:15:31.612558 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51802) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:31.618853 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.v2.base [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Request body: {'floatingip': {'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'port_id': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address': '172.20.0.4'}} {{(pid=99544) prepare_request_body /opt/stack/neutron/neutron/api/v2/base.py:729}} May 25 16:15:31.625269 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources subnetpool have unlimited quota limit. It is not required to calculate headroom {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:219}} May 25 16:15:31.629878 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Usage tracker for resource:floatingip and tenant:a1e31a264b7949bea6aa47cb7781106b is out of sync, need to count used quota {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:270}} May 25 16:15:31.632960 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.quota.resource [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Quota usage for floatingip was recalculated. Used quota:2. {{(pid=99544) count_used /opt/stack/neutron/neutron/quota/resource.py:290}} May 25 16:15:31.639570 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.quota.driver [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attempting to reserve 1 items for resource floatingip. Total usage: 2; quota limit: 50; headroom:48 {{(pid=99544) make_reservation /opt/stack/neutron/neutron/db/quota/driver.py:246}} May 25 16:15:31.735475 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:31.875902 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:31.930304 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-346cd122-6bb4-406a-bc3c-954ab98156a8 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:31.931834 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-346cd122-6bb4-406a-bc3c-954ab98156a8 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.42 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.0830493 May 25 16:15:32.029424 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_common [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Allocated IP 172.27.2.95 (0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc/9237fe36-57f4-4495-8b6e-e6960468c84c/21d6c459-68d0-4850-a0f5-071e55c8f47a) {{(pid=99544) _store_ip_allocation /opt/stack/neutron/neutron/db/db_base_plugin_common.py:131}} May 25 16:15:32.184915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Deleting port 145301a2-46f6-4ded-b871-52227a79d41a {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:32.185615 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.214709 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.339369 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] For port 145301a2-46f6-4ded-b871-52227a79d41a, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=145301a2-46f6-4ded-b871-52227a79d41a,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:32.339731 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.358445 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_new_port-8747111529903', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.364796 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6ece0bca-5c0b-4e6d-a6f2-c4f26145c341 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:32.432153 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.dhcp_agent_scheduler [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Network 0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc is already hosted by enough agents. {{(pid=99544) _get_dhcp_agents_hosting_network /opt/stack/neutron/neutron/scheduler/dhcp_agent_scheduler.py:275}} May 25 16:15:32.435120 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.base_scheduler [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Resources bound (agent type, host, resource id): {{(pid=99544) schedule /opt/stack/neutron/neutron/scheduler/base_scheduler.py:57}} May 25 16:15:32.439213 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Received port after_create {{(pid=99544) _notify_l3_agent_new_port /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:537}} May 25 16:15:32.475201 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=21d6c459-68d0-4850-a0f5-071e55c8f47a,revision_number=1']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:32.483941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.119s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:32.506906 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Calling delete_port for 145301a2-46f6-4ded-b871-52227a79d41a owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:32.574890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d0d83064-92bf-485d-a982-8c2a1bd025c1 req-476896a0-4930-489e-ad91-f4f33f41325c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:32.576423 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d83064-92bf-485d-a982-8c2a1bd025c1 req-476896a0-4930-489e-ad91-f4f33f41325c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 945 time: 0.1739545 May 25 16:15:32.668226 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d0d83064-92bf-485d-a982-8c2a1bd025c1 req-21b0c352-58cf-4d16-b69f-75058e5e5d73 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/security-groups?id=8286425c-b378-48e3-9125-58c869823a8e&fields=id&fields=name HTTP/1.1" status: 200 len: 286 time: 0.0837798 May 25 16:15:32.781126 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.788704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:32.815621 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] get_ports_and_sgs() called for port_ids ['145301a2-46f6-4ded-b871-52227a79d41a'] {{(pid=99546) get_ports_and_sgs /opt/stack/neutron/neutron/plugins/ml2/db.py:164}} May 25 16:15:32.837915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Pushing event deleted for resources: {'Port': ['ID=145301a2-46f6-4ded-b871-52227a79d41a,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:32.840502 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:32.881749 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_create {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.918778 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] No DVR routers for this DVR port 145301a2-46f6-4ded-b871-52227a79d41a on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:15:32.921412 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:32.924168 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'name': 'network-vif-deleted', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:32.936647 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-afbc0d83-3e75-47b4-b836-3417e3283004 req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.42 "DELETE /v2.0/ports/145301a2-46f6-4ded-b871-52227a79d41a HTTP/1.1" status: 204 len: 173 time: 0.9987154 May 25 16:15:32.969856 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '9513033a-aafc-4278-8620-b6c4dbdffe11', 'name': 'network-vif-deleted', 'tag': '145301a2-46f6-4ded-b871-52227a79d41a', 'status': 'completed', 'code': 200} May 25 16:15:33.039753 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler_port-16139537'] for port, before_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:33.269454 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for port, precommit_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:33.369323 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_gateway_port_ip_changed-8747111867540', 'neutron.db.l3_dvrscheduler_db._notify_l3_agent_port_update-8747111529921', 'neutron.db.l3_hascheduler_db._notify_l3_agent_ha_port_update-8747111529966', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_updated-3070946'] for port, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:33.373365 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-3a01437a-90c5-4dea-8671-b7d5c843d8e9 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:33.629170 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event updated for resources: {'Port': ['ID=21d6c459-68d0-4850-a0f5-071e55c8f47a,revision_number=2']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:33.633671 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.260s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:33.644268 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._create_dvr_floating_gw_port-490496'] for floatingip, after_update {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:33.827329 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP a5f69962-d2b8-43b2-862c-cddab359ed5f associated. External IP: 172.27.2.95, port: c3ce7b44-762a-4e22-a20f-491edca650be. May 25 16:15:33.884416 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-82cd6d8d-96c9-486e-983d-d63cf5bd4c4e req-304cea73-7354-476c-b18e-0851638386d0 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Synchronizing usage tracker for tenant:7413f95c7d284674bb0d587de6b42810 on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:15:33.980968 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-82cd6d8d-96c9-486e-983d-d63cf5bd4c4e req-304cea73-7354-476c-b18e-0851638386d0 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Unset dirty status for tenant:7413f95c7d284674bb0d587de6b42810 on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:15:33.982140 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-82cd6d8d-96c9-486e-983d-d63cf5bd4c4e req-304cea73-7354-476c-b18e-0851638386d0 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] 10.222.0.82 "GET /v2.0/ports?device_id=9513033a-aafc-4278-8620-b6c4dbdffe11 HTTP/1.1" status: 200 len: 210 time: 0.1853614 May 25 16:15:34.205986 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:15:34.391308 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:15:34.403592 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:15:34.414340 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:34.415320 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:34.479167 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:34.537724 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9ae519da-f02e-4913-9403-966cbb3e9111 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "POST /v2.0/floatingips HTTP/1.1" status: 201 len: 995 time: 2.9241107 May 25 16:15:34.538407 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:15:34.586151 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'status': 'completed', 'code': 200} May 25 16:15:34.601552 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:34.707881 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-367a7266-1345-4e38-99ba-75521c55e441 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:34.709340 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-367a7266-1345-4e38-99ba-75521c55e441 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1098 time: 0.0790265 May 25 16:15:34.847447 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-2795c4d1-5360-4851-8291-e34ac37e4639 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:34.849378 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-2795c4d1-5360-4851-8291-e34ac37e4639 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1335814 May 25 16:15:34.960929 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-7e3ab570-46e9-4e23-8487-9a20cabddf87 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 200 len: 998 time: 0.1050427 May 25 16:15:35.033495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-4905ce6f-219a-4629-a3c9-5ffc03fa9c64 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:35.035058 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-4905ce6f-219a-4629-a3c9-5ffc03fa9c64 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0687027 May 25 16:15:35.044332 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}, {'id': 'a5f69962-d2b8-43b2-862c-cddab359ed5f', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.95', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:64:d4:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58e18ce7-8323-44d8-aa49-6157273d7138', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:15:32Z', 'updated_at': '2021-05-25T16:15:32Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:35.051419 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:15:35.067384 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:35.068345 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:35.121997 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-6449a416-0587-42eb-9bb0-f7275f2347d9 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:15:35.123579 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-6449a416-0587-42eb-9bb0-f7275f2347d9 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0827427 May 25 16:15:35.273866 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-a9b6d916-4e08-4f3c-8ad5-5333ef4c14fd service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1435878 May 25 16:15:35.279503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:35.429627 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-ebbdf8e9-be3d-4675-aa65-afc0602451ac req-8e30dce6-b061-41cc-a809-104bcb25cd16 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1489842 May 25 16:15:35.659903 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}, {'id': 'a5f69962-d2b8-43b2-862c-cddab359ed5f', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.95', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:64:d4:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58e18ce7-8323-44d8-aa49-6157273d7138', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:15:32Z', 'updated_at': '2021-05-25T16:15:32Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:35.665102 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-d22137ba-cc4c-417b-9527-fedad34d6524 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}, {'id': 'a5f69962-d2b8-43b2-862c-cddab359ed5f', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.95', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'fixed_ip_address': '172.20.0.4', 'status': 'DOWN', 'standard_attr_id': 687, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:64:d4:d5', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '58e18ce7-8323-44d8-aa49-6157273d7138', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:15:32Z', 'updated_at': '2021-05-25T16:15:32Z', 'revision_number': 0, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:15:35.695380 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9f69d7fe-9a40-492e-a561-d7a16135cb45 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:35.867594 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-ff8d99d8-35b2-4de3-93f3-9f7337241e83 None None] New status for floating IP a5f69962-d2b8-43b2-862c-cddab359ed5f: ACTIVE {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:15:36.444961 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51862) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:36.571421 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 145301a2-46f6-4ded-b871-52227a79d41a no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:36.606843 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 145301a2-46f6-4ded-b871-52227a79d41a {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:36.607244 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 145301a2-46f6-4ded-b871-52227a79d41a not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:37.131720 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:37.555219 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Deleting port 61c3265e-fd7c-491c-8373-f3de5eca4718 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:37.555991 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:37.685899 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] For port 61c3265e-fd7c-491c-8373-f3de5eca4718, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=61c3265e-fd7c-491c-8373-f3de5eca4718,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:37.686251 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:37.813594 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Calling delete_port for 61c3265e-fd7c-491c-8373-f3de5eca4718 owned by network:router_interface {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:38.152425 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:38.156584 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-03aa573b-f5e6-4dfa-bd75-35b09df97d9a admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:38.165016 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:38.204230 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Pushing event deleted for resources: {'Port': ['ID=61c3265e-fd7c-491c-8373-f3de5eca4718,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:38.208704 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.052s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:38.438505 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-55a1ffa0-2d43-4bb4-b660-8c080ef92f5a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:38.514314 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:39.057121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Router 0492512e-ccab-4c81-9788-1e5b615f9dc3 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:15:39.130600 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:39.261311 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:15:39.274208 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Sync routers for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:15:39.286074 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:39.287283 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:39.361071 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-67f3960a-c336-4b1f-8498-7a8f17c077f5 admin admin] 10.222.0.82 "PUT /v2.0/routers/0492512e-ccab-4c81-9788-1e5b615f9dc3/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9142945 May 25 16:15:39.364410 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51866) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:39.550316 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:39.791947 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks [] for router, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:39.885332 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:15:38Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:39.890180 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Checking router: 0492512e-ccab-4c81-9788-1e5b615f9dc3 for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:15:39.900514 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Routers {'0492512e-ccab-4c81-9788-1e5b615f9dc3'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:39.901176 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['0492512e-ccab-4c81-9788-1e5b615f9dc3'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:40.009227 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:40.149193 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] Set router 0492512e-ccab-4c81-9788-1e5b615f9dc3 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:40.156072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Deleting port 199437ae-99fb-4810-a6e7-6003a7272edd {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:40.156426 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:40.296517 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] For port 199437ae-99fb-4810-a6e7-6003a7272edd, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=199437ae-99fb-4810-a6e7-6003a7272edd,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:40.296915 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:40.427305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Calling delete_port for 199437ae-99fb-4810-a6e7-6003a7272edd owned by network:router_gateway {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:40.450630 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:15:38Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:40.455342 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-401fb2ab-ae50-4b2b-b0f4-fbde3f228f26 None None] The sync data for ids ['0492512e-ccab-4c81-9788-1e5b615f9dc3'] in dv-cinder-786384-6-iscsi is: [{'id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'name': 'tempest-TestMinimumBasicScenario-1030053623-router', 'tenant_id': '7413f95c7d284674bb0d587de6b42810', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138'}], 'enable_snat': True}, 'gw_port_id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:03Z', 'updated_at': '2021-05-25T16:15:38Z', 'revision_number': 5, 'project_id': '7413f95c7d284674bb0d587de6b42810', 'gw_port': {'id': '199437ae-99fb-4810-a6e7-6003a7272edd', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:e5:ce:fb', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '0492512e-ccab-4c81-9788-1e5b615f9dc3', 'device_owner': 'network:router_gateway', 'standard_attr_id': 656, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.138', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:04Z', 'updated_at': '2021-05-25T16:13:12Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:15:40.558095 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:40.820911 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:40.826850 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:40.874125 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Pushing event deleted for resources: {'Port': ['ID=199437ae-99fb-4810-a6e7-6003a7272edd,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:40.877330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:40.979549 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.140554 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.170917 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.173716 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.290364 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.291361 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Fanout notify agent at l3_agent the message router_deleted on router 0492512e-ccab-4c81-9788-1e5b615f9dc3 {{(pid=99545) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:15:41.310231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.311566 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2c159f8a-1093-4789-a971-35d1b859a04e admin admin] 10.222.0.82 "DELETE /v2.0/routers/0492512e-ccab-4c81-9788-1e5b615f9dc3 HTTP/1.1" status: 204 len: 168 time: 1.9456370 May 25 16:15:41.314690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51872) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:41.407914 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Deleting subnet 70734104-38bb-429b-abc3-63fbc11e4122 {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:15:41.412073 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.470166 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:41.697649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.084119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.088434 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:42.126038 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Pushing event deleted for resources: {'Subnet': ['ID=70734104-38bb-429b-abc3-63fbc11e4122,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:42.129796 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.041s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:42.131114 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.152476 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-af7f8c5f-fda1-4819-b5f4-eb14efe8e974 None None] Network ceb9e841-143d-468e-800d-756d578d900d requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:15:42.153582 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-499b25ed-7715-4b66-9999-3009b6d64edb admin admin] 10.222.0.82 "DELETE /v2.0/subnets/70734104-38bb-429b-abc3-63fbc11e4122 HTTP/1.1" status: 204 len: 168 time: 0.8303683 May 25 16:15:42.154363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51878) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:42.236391 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 199437ae-99fb-4810-a6e7-6003a7272edd no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:42.285237 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.289888 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 199437ae-99fb-4810-a6e7-6003a7272edd {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:42.290234 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 199437ae-99fb-4810-a6e7-6003a7272edd not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:42.290656 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 61c3265e-fd7c-491c-8373-f3de5eca4718 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:42.340838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 61c3265e-fd7c-491c-8373-f3de5eca4718 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:42.341190 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 61c3265e-fd7c-491c-8373-f3de5eca4718 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:42.393273 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Deleting port b8e1535c-25c9-469e-8c50-6b96e5f67f74 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:42.393590 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.508274 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] For port b8e1535c-25c9-469e-8c50-6b96e5f67f74, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b8e1535c-25c9-469e-8c50-6b96e5f67f74,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:42.509358 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.562626 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] DHCP port deletion for ceb9e841-143d-468e-800d-756d578d900d request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:15:42.649873 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Deleting port b8e1535c-25c9-469e-8c50-6b96e5f67f74 {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:42.650257 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.664633 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Calling delete_port for b8e1535c-25c9-469e-8c50-6b96e5f67f74 owned by network:dhcp {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:42.810113 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.815388 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-902b8c1c-02f8-434b-87cd-39cf97b47f5c tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:42.824162 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] For port b8e1535c-25c9-469e-8c50-6b96e5f67f74, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=b8e1535c-25c9-469e-8c50-6b96e5f67f74,segment=NetworkSegment(6075f059-4c81-4008-876b-49bbdcfe295e),segment_id=6075f059-4c81-4008-876b-49bbdcfe295e)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:42.824576 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:42.860400 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Pushing event deleted for resources: {'Port': ['ID=b8e1535c-25c9-469e-8c50-6b96e5f67f74,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:42.864200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:42.864855 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] No DVR routers for this DVR port b8e1535c-25c9-469e-8c50-6b96e5f67f74 on host dv-cinder-786384-6-iscsi {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:15:42.960924 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Calling delete_port for b8e1535c-25c9-469e-8c50-6b96e5f67f74 owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:42.989515 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.011890 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['ceb9e841-143d-468e-800d-756d578d900d']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:43.012790 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.016740 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:43.036912 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '6075f059-4c81-4008-876b-49bbdcfe295e') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:43.037405 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '6075f059-4c81-4008-876b-49bbdcfe295e') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:43.056041 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.062035 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Pushing event deleted for resources: {'Port': ['ID=b8e1535c-25c9-469e-8c50-6b96e5f67f74,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:43.065329 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:43.066236 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-58700ad0-7974-4ef9-b0cd-eea4e34a0aa0 None None] No DVR routers for this DVR port b8e1535c-25c9-469e-8c50-6b96e5f67f74 on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:15:43.082503 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.090093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Releasing vlan 916 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:15:43.090603 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:15:43.106423 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:15:43.244169 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.247911 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-077ab7ac-1521-4de8-a117-ee8677a34aa3 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:43.261642 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:43.273484 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Pushing event deleted for resources: {'Network': ['ID=ceb9e841-143d-468e-800d-756d578d900d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:43.277040 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.029s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:43.369017 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-2f345881-3f84-4c16-8e11-fb7a264cefec admin admin] 10.222.0.82 "DELETE /v2.0/networks/ceb9e841-143d-468e-800d-756d578d900d HTTP/1.1" status: 204 len: 168 time: 1.2086103 May 25 16:15:43.369017 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51882) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:44.046959 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:44.196488 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b8e1535c-25c9-469e-8c50-6b96e5f67f74 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:44.257355 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: b8e1535c-25c9-469e-8c50-6b96e5f67f74 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:44.257859 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device b8e1535c-25c9-469e-8c50-6b96e5f67f74 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:44.516990 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Deleting port bef09144-4688-468a-b1ed-5f1bf3929ba3 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:44.517324 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:44.653135 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] For port bef09144-4688-468a-b1ed-5f1bf3929ba3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=bef09144-4688-468a-b1ed-5f1bf3929ba3,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:44.653494 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:44.790048 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Calling delete_port for bef09144-4688-468a-b1ed-5f1bf3929ba3 owned by network:router_interface {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:45.026970 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:45.031065 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0fb0f565-9985-42c8-9064-d0ae5927720e service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:45.079733 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Pushing event deleted for resources: {'Port': ['ID=bef09144-4688-468a-b1ed-5f1bf3929ba3,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:45.084076 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.053s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:45.477119 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:45.860368 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Router acd58228-b46f-4a8e-be41-0910702cff46 has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:15:46.047715 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:15:46.057447 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Sync routers for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:15:46.068377 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:46.069001 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:46.137501 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-9f54a5e7-c620-4be2-9dfe-c6f3588a0fd8 admin admin] 10.222.0.82 "PUT /v2.0/routers/acd58228-b46f-4a8e-be41-0910702cff46/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.7685146 May 25 16:15:46.140090 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51888) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:46.287584 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:46.375302 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks [] for router, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:46.604231 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:46.643313 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164'}], 'enable_snat': True}, 'gw_port_id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:15:44Z', 'revision_number': 5, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port': {'id': 'baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:85:98:6a', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'device_owner': 'network:router_gateway', 'standard_attr_id': 669, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.164', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:13:15Z', 'updated_at': '2021-05-25T16:13:22Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:46.647096 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Checking router: acd58228-b46f-4a8e-be41-0910702cff46 for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:15:46.667286 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Routers {'acd58228-b46f-4a8e-be41-0910702cff46'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:15:46.667897 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 5, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 5, "log_agent_heartbeats": false, "routers": 5},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:15:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['acd58228-b46f-4a8e-be41-0910702cff46'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:46.751117 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Deleting port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:46.751461 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:46.829983 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] Set router acd58228-b46f-4a8e-be41-0910702cff46 gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:15:46.892330 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] For port baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:46.892683 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:46.945401 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:15:46Z', 'revision_number': 6, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:46.949751 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-73525c67-a7b3-4f31-ad63-5ea41ad31cd2 None None] The sync data for ids ['acd58228-b46f-4a8e-be41-0910702cff46'] in dv-cinder-786384-6-iscsi is: [{'id': 'acd58228-b46f-4a8e-be41-0910702cff46', 'name': 'tempest-TestMinimumBasicScenario-299703303-router', 'tenant_id': '92165e91edbb47c88928d15e0da2d7bd', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:13:14Z', 'updated_at': '2021-05-25T16:15:46Z', 'revision_number': 6, 'project_id': '92165e91edbb47c88928d15e0da2d7bd', 'gw_port_host': None}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:15:47.024388 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Calling delete_port for baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 owned by network:router_gateway {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:47.388404 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.391333 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-faa0df74-4c0a-435b-94c6-2fcec3f70001 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:47.438854 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Pushing event deleted for resources: {'Port': ['ID=baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:47.442092 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.051s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:47.535702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.748702 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.751699 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.843780 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.844304 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Fanout notify agent at l3_agent the message router_deleted on router acd58228-b46f-4a8e-be41-0910702cff46 {{(pid=99545) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:15:47.858206 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.859462 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-21699ef0-c57e-4e32-9f18-54337a0f9269 admin admin] 10.222.0.82 "DELETE /v2.0/routers/acd58228-b46f-4a8e-be41-0910702cff46 HTTP/1.1" status: 204 len: 168 time: 1.7182126 May 25 16:15:47.862274 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51892) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:47.927018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Deleting subnet d4ddf00b-460c-41c8-8051-1d52a3935d5b {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:15:47.930725 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:47.993746 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.075709 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.185793 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bef09144-4688-468a-b1ed-5f1bf3929ba3 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:48.221533 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: bef09144-4688-468a-b1ed-5f1bf3929ba3 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:48.221884 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device bef09144-4688-468a-b1ed-5f1bf3929ba3 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:48.222337 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:48.256687 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:48.256950 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device baa30c69-ce1e-4242-ac2c-8e70aaa9dbe5 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:48.438471 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.443127 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:48.477841 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Pushing event deleted for resources: {'Subnet': ['ID=d4ddf00b-460c-41c8-8051-1d52a3935d5b,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:48.480885 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.038s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:48.481924 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.487438 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-3c615005-bf76-4b21-bad4-2e008fbe5a64 None None] Network 4ae9aef2-8f6b-4c56-b394-f86738d1b047 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:15:48.488733 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-436293c3-96d9-49b5-88b5-39432f0db38d admin admin] 10.222.0.82 "DELETE /v2.0/subnets/d4ddf00b-460c-41c8-8051-1d52a3935d5b HTTP/1.1" status: 204 len: 168 time: 0.6249511 May 25 16:15:48.493426 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51898) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:48.618945 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.702609 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Deleting port 05f35bc2-759b-4098-ab8d-b513e2a36c4d {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:48.702989 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.749805 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] DHCP port deletion for 4ae9aef2-8f6b-4c56-b394-f86738d1b047 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:15:48.812097 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] For port 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=05f35bc2-759b-4098-ab8d-b513e2a36c4d,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:48.812447 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.819345 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Deleting port 05f35bc2-759b-4098-ab8d-b513e2a36c4d {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:15:48.819729 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:48.986601 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Calling delete_port for 05f35bc2-759b-4098-ab8d-b513e2a36c4d owned by network:dhcp {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:48.991510 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] For port 05f35bc2-759b-4098-ab8d-b513e2a36c4d, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=05f35bc2-759b-4098-ab8d-b513e2a36c4d,segment=NetworkSegment(22130cb2-3662-4892-b32e-70b172a78d26),segment_id=22130cb2-3662-4892-b32e-70b172a78d26)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:15:48.992157 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.124165 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.129697 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a47cb037-49d6-433c-a44d-e95cfa3ba52f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:49.166648 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Calling delete_port for 05f35bc2-759b-4098-ab8d-b513e2a36c4d owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:15:49.182506 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Pushing event deleted for resources: {'Port': ['ID=05f35bc2-759b-4098-ab8d-b513e2a36c4d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:49.185643 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] No DVR routers for this DVR port 05f35bc2-759b-4098-ab8d-b513e2a36c4d on host dv-cinder-786384-6-iscsi {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:15:49.188339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.059s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:49.236308 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.243077 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:49.294352 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Pushing event deleted for resources: {'Port': ['ID=05f35bc2-759b-4098-ab8d-b513e2a36c4d,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:49.296965 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.054s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:49.299811 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-c3059a23-7e60-40ab-a7cb-7bf036433fff None None] No DVR routers for this DVR port 05f35bc2-759b-4098-ab8d-b513e2a36c4d on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:15:49.427495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.444815 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['4ae9aef2-8f6b-4c56-b394-f86738d1b047']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:49.458341 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '22130cb2-3662-4892-b32e-70b172a78d26') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:49.458635 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '22130cb2-3662-4892-b32e-70b172a78d26') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:15:49.470284 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.488024 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.494331 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Releasing vlan 933 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:15:49.494658 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:15:49.508554 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:15:49.649563 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.653006 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:49.671239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:49.683845 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Pushing event deleted for resources: {'Network': ['ID=4ae9aef2-8f6b-4c56-b394-f86738d1b047,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:49.687805 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.035s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:49.690356 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-d094d1e7-ec04-453c-9af0-8c01d7c36c24 admin admin] 10.222.0.82 "DELETE /v2.0/networks/4ae9aef2-8f6b-4c56-b394-f86738d1b047 HTTP/1.1" status: 204 len: 168 time: 1.1948714 May 25 16:15:50.146823 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51904) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:50.189933 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 05f35bc2-759b-4098-ab8d-b513e2a36c4d no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:15:50.208996 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8b2a7e2f-2594-4e9b-a1dd-6a9adb19c448 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=7413f95c7d284674bb0d587de6b42810&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0608330 May 25 16:15:50.211620 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51908) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:50.229055 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 05f35bc2-759b-4098-ab8d-b513e2a36c4d {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:15:50.229413 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 05f35bc2-759b-4098-ab8d-b513e2a36c4d not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:15:50.282712 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:50.311997 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:50.455248 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:50.459134 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-4be33ce9-d0b2-444c-b8c1-c1c2a24642bd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:50.474220 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=8286425c-b378-48e3-9125-58c869823a8e,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:50.476783 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.018s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:50.483420 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:50.484553 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-e1bc6e1e-9828-4a2b-91fe-da8a8e89beb3 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/8286425c-b378-48e3-9125-58c869823a8e HTTP/1.1" status: 204 len: 168 time: 0.2720304 May 25 16:15:51.241131 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51916) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:51.294277 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-dbbaceb5-73f7-40f7-bd1a-c037bbfe003f admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=92165e91edbb47c88928d15e0da2d7bd&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0518208 May 25 16:15:51.296925 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51920) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:15:51.359495 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:51.386856 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:51.529425 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:51.533310 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-b0ac2465-bfab-45dd-97f4-b345ec2ff99a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:15:51.544744 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=6b980970-f336-4735-a014-d970af478641,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:15:51.547351 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.014s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:15:51.555854 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:15:51.557043 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5029d302-a6aa-4c4f-bcac-eb2600824821 admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/6b980970-f336-4735-a014-d970af478641 HTTP/1.1" status: 204 len: 168 time: 0.2591918 May 25 16:15:53.302277 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:16:05.666381 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-1342b21e-d02c-4aa4-a49a-81cd3c673add None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:08.152204 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:08.426315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4dbfb088-408a-4954-8e18-b86c1b57f3b1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:09.110893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:10.574762 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:11.128899 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:16.945628 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-e5789972-8c68-4760-879d-8641de6f3746 service neutron] 10.222.0.42 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884&fields=binding%3Ahost_id&fields=binding%3Avif_type HTTP/1.1" status: 200 len: 275 time: 0.0748343 May 25 16:16:17.106046 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-0f8b57f0-474e-4051-842a-a9df877e1e3f service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:16:17.107543 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-0f8b57f0-474e-4051-842a-a9df877e1e3f service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1098 time: 0.0715361 May 25 16:16:17.255126 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-d6f2de45-212a-4058-b4ee-c46aaeffaa4a service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:16:17.256388 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-d6f2de45-212a-4058-b4ee-c46aaeffaa4a service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1294456 May 25 16:16:17.361309 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-eefd36b4-0b7a-411d-94f0-6477a565247d service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 1000 time: 0.0980079 May 25 16:16:17.425003 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-ab71e165-97ee-4ca2-a372-db56d2dfe6ba service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:16:17.426547 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-ab71e165-97ee-4ca2-a372-db56d2dfe6ba service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0574908 May 25 16:16:17.504526 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-5d02cc69-a858-4dee-8990-cb49c8c47686 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:16:17.506025 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-5d02cc69-a858-4dee-8990-cb49c8c47686 service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.0737176 May 25 16:16:17.645598 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-494a4120-90a6-4863-aa86-f7d5146a5b9a service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1324973 May 25 16:16:17.783360 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-0adc050b-a5fe-437b-9a7c-2599fb1d62d5 req-e3966b1f-2765-46d9-be5b-b3f19e44592c service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1312525 May 25 16:16:30.301437 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:16:35.671284 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-a44155e0-d65f-4aba-badf-947a7610a889 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:38.128583 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:38.442748 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c17cfc9d-c739-4446-91a9-000fa2f9cc68 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:39.139887 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:40.586043 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:16:41.134367 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:02.069533 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 51952) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:02.344095 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port 21d6c459-68d0-4850-a0f5-071e55c8f47a {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:02.344339 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:02.464753 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:02.465085 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:02.587766 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for 21d6c459-68d0-4850-a0f5-071e55c8f47a owned by network:floatingip {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:02.785465 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:02.789375 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:02.833106 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=21d6c459-68d0-4850-a0f5-071e55c8f47a,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:02.870374 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.870794 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.871235 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.871650 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.872031 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.872427 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.872821 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.873211 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.873607 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.873995 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.874460 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.874923 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99544) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:02.883148 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.094s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:03.050590 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:03.050919 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.db.l3_db [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP a5f69962-d2b8-43b2-862c-cddab359ed5f disassociated (deleted). External IP: 172.27.2.95, port: c3ce7b44-762a-4e22-a20f-491edca650be. May 25 16:17:03.413310 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99544) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:17:03.598443 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99544) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:17:03.608836 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:17:03.618175 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:03.618799 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:16:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:03.746066 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:03.801493 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:03.802009 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-8976d032-c9a5-4769-9f33-ec5035ea43bb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "DELETE /v2.0/floatingips/a5f69962-d2b8-43b2-862c-cddab359ed5f HTTP/1.1" status: 204 len: 168 time: 1.7311409 May 25 16:17:03.802631 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be'}] {{(pid=99544) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:17:03.855065 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'status': 'completed', 'code': 200} May 25 16:17:03.974254 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-17d14110-ca07-499a-a0cb-a5d01083c8d6 service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:03.976119 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-17d14110-ca07-499a-a0cb-a5d01083c8d6 service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 1098 time: 0.0791037 May 25 16:17:04.110608 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-452c20f0-972b-47cf-904c-882bb1241144 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:04.112087 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-452c20f0-972b-47cf-904c-882bb1241144 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.1296973 May 25 16:17:04.184659 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-8c3be8bf-2bdb-4b76-89e4-2edc845a7587 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.4&port_id=c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 200 len: 217 time: 0.0663602 May 25 16:17:04.188114 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:04.192339 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:17:04.201207 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:04.202114 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 2, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:16:38Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:04.248072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-7943612f-f944-4887-82d2-3fa1948fbf83 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:04.249533 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-7943612f-f944-4887-82d2-3fa1948fbf83 service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.0594087 May 25 16:17:04.392268 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:04.450301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-3c7c7267-0777-4be5-ad0e-05adafe0e03d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:04.452164 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-3c7c7267-0777-4be5-ad0e-05adafe0e03d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1965163 May 25 16:17:04.651544 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-2e528270-8dfd-4036-9f96-564900f97866 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1916192 May 25 16:17:04.663458 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-6e30bf82-4393-47aa-bd9b-958b85587713 req-41cbc021-d393-4c39-8ca8-cc0c2e84b4dc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:04.664925 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e30bf82-4393-47aa-bd9b-958b85587713 req-41cbc021-d393-4c39-8ca8-cc0c2e84b4dc tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.1341772 May 25 16:17:04.792057 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-6e30bf82-4393-47aa-bd9b-958b85587713 req-cc9a2e2f-1fe1-4de4-a1b5-596a23377ebb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.1076152 May 25 16:17:04.800503 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:04.803264 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-2b454920-7a8e-40a3-8b0a-927d9577cddc None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_floatingips': [{'id': '6b0ddb8d-7792-4174-bb8b-673891182098', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'floating_ip_address': '172.27.2.21', 'floating_network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'router_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'port_id': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'fixed_ip_address': '172.20.0.5', 'status': 'ACTIVE', 'standard_attr_id': 672, 'description': '', 'qos_policy_id': None, 'port_details': {'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'mac_address': 'fa:16:3e:5c:31:d1', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'device_owner': 'compute:nova'}, 'dns_domain': '', 'dns_name': '', 'tags': [], 'created_at': '2021-05-25T16:13:28Z', 'updated_at': '2021-05-25T16:13:32Z', 'revision_number': 1, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'fixed_ip_address_scope': None}], '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:17:04.836196 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-83fa8540-9c8c-40d6-8047-b2558391d6bd req-33ad6057-eb81-4ce4-8417-1d911cb4b86e service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1703329 May 25 16:17:05.013657 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-4f82da86-e62d-4b92-87c8-ee516d065aa1 None None] New status for floating IP a5f69962-d2b8-43b2-862c-cddab359ed5f: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:17:05.654052 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-c83bc2d8-bc87-4d35-a650-d65a793587d6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:05.993351 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-2bdb428e-c188-4002-baae-df1430675712 req-f260a11f-8939-431c-b771-82b8237d0ab2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:05.994708 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2bdb428e-c188-4002-baae-df1430675712 req-f260a11f-8939-431c-b771-82b8237d0ab2 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.0802262 May 25 16:17:06.087463 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2bdb428e-c188-4002-baae-df1430675712 req-9e12fe02-e1d7-402e-bbdb-9b10cfa6c421 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0869591 May 25 16:17:06.117469 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-856b6ad3-87a5-4a48-a649-51d250af238c req-4a6e98aa-aed5-4c10-a475-d48ca39592eb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:06.118960 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-856b6ad3-87a5-4a48-a649-51d250af238c req-4a6e98aa-aed5-4c10-a475-d48ca39592eb tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 945 time: 0.0987406 May 25 16:17:06.258074 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port c3ce7b44-762a-4e22-a20f-491edca650be {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:06.258308 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:06.391556 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] For port c3ce7b44-762a-4e22-a20f-491edca650be, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=c3ce7b44-762a-4e22-a20f-491edca650be,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:06.391908 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:06.514484 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for c3ce7b44-762a-4e22-a20f-491edca650be owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:06.697886 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:06.704002 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-4a61e6e6-b148-40c0-a8b7-2260b6923e98 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:06.752453 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=c3ce7b44-762a-4e22-a20f-491edca650be,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:06.791698 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.792077 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.792449 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.792814 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.793174 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.793533 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.793895 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.794249 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.794604 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.795100 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.795464 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.795817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99545) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:06.803072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.099s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:06.842900 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] No DVR routers for this DVR port c3ce7b44-762a-4e22-a20f-491edca650be on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:06.845553 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:06.848500 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'name': 'network-vif-deleted', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:17:06.860834 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-856b6ad3-87a5-4a48-a649-51d250af238c req-bf337bf1-d63c-4228-8c43-7cd814a80f96 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "DELETE /v2.0/ports/c3ce7b44-762a-4e22-a20f-491edca650be HTTP/1.1" status: 204 len: 173 time: 0.7368126 May 25 16:17:06.901454 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '58e18ce7-8323-44d8-aa49-6157273d7138', 'name': 'network-vif-deleted', 'tag': 'c3ce7b44-762a-4e22-a20f-491edca650be', 'status': 'completed', 'code': 200} May 25 16:17:07.247037 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-2830adda-12da-48ee-b702-b9530ac6db63 req-ab6d2d7f-0048-4891-9df6-e48def6ee902 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:17:07.300523 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:17:07.308951 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-2830adda-12da-48ee-b702-b9530ac6db63 req-ab6d2d7f-0048-4891-9df6-e48def6ee902 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:17:07.310104 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-2830adda-12da-48ee-b702-b9530ac6db63 req-ab6d2d7f-0048-4891-9df6-e48def6ee902 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 210 time: 0.1067233 May 25 16:17:08.191229 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:08.420058 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-939f7069-7c03-40fa-abb6-a5261d2f302d None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:08.482152 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-edf5d7d7-b648-487b-aec4-6afa69608f60 req-777df108-7e7a-4605-87b8-dbcf3997394a tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=58e18ce7-8323-44d8-aa49-6157273d7138 HTTP/1.1" status: 200 len: 210 time: 0.0434525 May 25 16:17:08.990727 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device c3ce7b44-762a-4e22-a20f-491edca650be no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:09.028948 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: c3ce7b44-762a-4e22-a20f-491edca650be {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:09.029290 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device c3ce7b44-762a-4e22-a20f-491edca650be not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:09.137828 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:10.602441 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:11.173949 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:12.239339 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 51986) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:12.617000 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port 3c840498-7e4d-4fb0-bf72-4d3d2dfe855d {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:12.617340 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:12.740171 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:12.740511 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:12.886781 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for 3c840498-7e4d-4fb0-bf72-4d3d2dfe855d owned by network:floatingip {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:13.094091 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:13.098546 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-8c42eb10-5fbe-4a53-89ce-144d56f8e1ca service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:13.145305 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=3c840498-7e4d-4fb0-bf72-4d3d2dfe855d,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:13.148534 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:13.249059 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:13.249810 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.db.l3_db [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Floating IP 6b0ddb8d-7792-4174-bb8b-673891182098 disassociated (deleted). External IP: 172.27.2.21, port: 4b64460a-d17d-4ac3-82d0-d9990d9be8b4. May 25 16:17:13.629798 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:17:13.834151 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:17:13.843984 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:17:13.853693 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:13.854618 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:13.916718 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:13.982142 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-0c5555f1-2188-448c-b1ab-38b3edd3f4e8 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "DELETE /v2.0/floatingips/6b0ddb8d-7792-4174-bb8b-673891182098 HTTP/1.1" status: 204 len: 168 time: 1.7416203 May 25 16:17:13.982803 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:17:14.050392 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'name': 'network-changed', 'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'status': 'completed', 'code': 200} May 25 16:17:14.053260 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:14.199293 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-9a5bbd66-ee10-4b94-86b7-ecae37f23cda service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:14.199293 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-9a5bbd66-ee10-4b94-86b7-ecae37f23cda service neutron] 10.222.0.42 "GET /v2.0/ports?tenant_id=a1e31a264b7949bea6aa47cb7781106b&device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 1098 time: 0.0956531 May 25 16:17:14.456283 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-a9d20b28-346d-47c6-8c76-7dca04e2ee09 service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'vlan_transparent'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:14.458023 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-a9d20b28-346d-47c6-8c76-7dca04e2ee09 service neutron] 10.222.0.42 "GET /v2.0/networks?id=6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 200 len: 912 time: 0.2604470 May 25 16:17:14.504775 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:14.509849 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:17:14.519987 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:14.520983 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:14.581811 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-a4bcdfd4-25da-43cf-95c3-244a2ebac6a0 service neutron] 10.222.0.42 "GET /v2.0/floatingips?fixed_ip_address=172.20.0.5&port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 200 len: 217 time: 0.1160355 May 25 16:17:14.717730 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:14.733312 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-d9aaccb3-f68a-4c5a-b166-e376bbb1223d service neutron] Attributes excluded by policy engine: ['standard_attr_id', 'shared', 'segment_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:14.735888 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-d9aaccb3-f68a-4c5a-b166-e376bbb1223d service neutron] 10.222.0.42 "GET /v2.0/subnets?id=a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 200 len: 850 time: 0.1471078 May 25 16:17:14.871415 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1d86017b-7722-46b4-83e5-81b358838a08 req-5075c800-8327-4e69-9f51-44415754aba4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:14.872719 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-1d86017b-7722-46b4-83e5-81b358838a08 req-5075c800-8327-4e69-9f51-44415754aba4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:17:14.909726 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-f0455d08-0cd0-4172-adfb-8205b5b5e31d service neutron] Attributes excluded by policy engine: ['standard_attr_id'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:14.910270 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-f0455d08-0cd0-4172-adfb-8205b5b5e31d service neutron] 10.222.0.42 "GET /v2.0/ports?network_id=6c61cff6-814b-4af0-8599-e026b705cc35&device_owner=network%3Adhcp HTTP/1.1" status: 200 len: 1175 time: 0.1668382 May 25 16:17:14.995960 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-1d86017b-7722-46b4-83e5-81b358838a08 req-5075c800-8327-4e69-9f51-44415754aba4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:17:14.997093 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d86017b-7722-46b4-83e5-81b358838a08 req-5075c800-8327-4e69-9f51-44415754aba4 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.4646001 May 25 16:17:15.091745 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1d86017b-7722-46b4-83e5-81b358838a08 req-abfe4409-3a65-413d-9ce4-8fcb51326d05 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0797937 May 25 16:17:15.109200 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-d448521b-1227-4e09-9356-490dc8bb89b1 service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=segments HTTP/1.1" status: 200 len: 212 time: 0.1906564 May 25 16:17:15.204290 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:15.210034 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-e9d4d17d-6f88-4fdf-b1f2-2de8007aa1e4 None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:09:46Z', 'revision_number': 4, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi', '_interfaces': [{'id': '921d6ca2-8279-4d74-8ee2-789091baf141', 'name': '', 'network_id': '6c61cff6-814b-4af0-8599-e026b705cc35', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'mac_address': 'fa:16:3e:43:9e:10', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_interface', 'standard_attr_id': 516, 'fixed_ips': [{'subnet_id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'ip_address': '172.20.0.1', 'prefixlen': 28}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:45Z', 'updated_at': '2021-05-25T16:09:54Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'subnets': [{'id': 'a4d8980e-d38c-4165-af50-ed2f39dc708c', 'cidr': '172.20.0.0/28', 'gateway_ip': '172.20.0.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}]}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:17:15.279767 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-4e57b4f3-7ec8-423d-8fa5-041b10f6f5e6 req-3605d380-98ef-4eb9-82e8-8e84f24c866d service neutron] 10.222.0.42 "GET /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35?fields=provider%3Aphysical_network&fields=provider%3Anetwork_type HTTP/1.1" status: 200 len: 281 time: 0.1636915 May 25 16:17:15.455176 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-620a857a-726e-453a-a5da-af88bda89488 None None] New status for floating IP 6b0ddb8d-7792-4174-bb8b-673891182098: DOWN {{(pid=99547) update_floatingip_statuses /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:270}} May 25 16:17:16.321262 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-d4b2353a-9d26-4863-9d6a-efb6420b4f0b tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:16.323066 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-d4b2353a-9d26-4863-9d6a-efb6420b4f0b tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.1951764 May 25 16:17:16.464600 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.pecan_wsgi.hooks.policy_enforcement [req-d062c7c7-6001-48fc-8544-01613414956f req-f8516129-6d57-4583-9179-351ffca26875 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Attributes excluded by policy engine: ['standard_attr_id', 'binding:profile', 'binding:host_id', 'binding:vif_type', 'binding:vif_details'] {{(pid=99545) _exclude_attributes_by_policy /opt/stack/neutron/neutron/pecan_wsgi/hooks/policy_enforcement.py:255}} May 25 16:17:16.466254 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d062c7c7-6001-48fc-8544-01613414956f req-f8516129-6d57-4583-9179-351ffca26875 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 945 time: 0.2201498 May 25 16:17:16.530151 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-d062c7c7-6001-48fc-8544-01613414956f req-087b3780-7102-4722-9d7f-bbe2220b1f5d tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/security-groups?id=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64&fields=id&fields=name HTTP/1.1" status: 200 len: 312 time: 0.0527065 May 25 16:17:16.540518 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:16.540857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:16.684607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] For port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4, host hv-cinder-78638, got binding levels [PortBindingLevel(driver='hyperv',host='hv-cinder-78638',level=0,port_id=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:16.684965 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:16.827686 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 owned by compute:nova {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:17.025813 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:17.032985 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-a92172cf-6ab1-4def-8334-9b8b9fd3d559 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:17.084690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=4b64460a-d17d-4ac3-82d0-d9990d9be8b4,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:17.084690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:17.168749 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] No DVR routers for this DVR port 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 on host hv-cinder-78638 {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:17.171466 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for port, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:17.175363 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.notifiers.nova [-] Sending events: [{'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'name': 'network-vif-deleted', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4'}] {{(pid=99545) send_events /opt/stack/neutron/neutron/notifiers/nova.py:262}} May 25 16:17:17.223544 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.notifiers.nova [-] Nova event response: {'server_uuid': '21648d95-6f96-4bc1-ad78-832a1c05b884', 'name': 'network-vif-deleted', 'tag': '4b64460a-d17d-4ac3-82d0-d9990d9be8b4', 'status': 'completed', 'code': 200} May 25 16:17:17.352193 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-1453ab7e-a710-4f4b-886a-67122ff0d8d1 req-3d1a46a0-7444-4fbe-aad9-7c6ee26c18f1 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.42 "DELETE /v2.0/ports/4b64460a-d17d-4ac3-82d0-d9990d9be8b4 HTTP/1.1" status: 204 len: 173 time: 1.0222909 May 25 16:17:17.720202 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c1e73b0c-f55f-4f30-bc90-ac056436a968 req-b58da8f7-ca5c-45f9-974f-2cf92a89ce77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Synchronizing usage tracker for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) resync /opt/stack/neutron/neutron/quota/resource.py:241}} May 25 16:17:17.819857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.quota.resource [req-c1e73b0c-f55f-4f30-bc90-ac056436a968 req-b58da8f7-ca5c-45f9-974f-2cf92a89ce77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Unset dirty status for tenant:a1e31a264b7949bea6aa47cb7781106b on resource:port {{(pid=99545) _resync /opt/stack/neutron/neutron/quota/resource.py:233}} May 25 16:17:17.821420 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [req-c1e73b0c-f55f-4f30-bc90-ac056436a968 req-b58da8f7-ca5c-45f9-974f-2cf92a89ce77 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "GET /v2.0/ports?device_id=21648d95-6f96-4bc1-ad78-832a1c05b884 HTTP/1.1" status: 200 len: 210 time: 0.1474204 May 25 16:17:18.935424 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52004) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:19.084538 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 no longer exists at agent hyperv_hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:19.124416 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] No Port match for: 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:19.124691 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Device 4b64460a-d17d-4ac3-82d0-d9990d9be8b4 not bound to the agent host hv-cinder-78638 {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:19.272747 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Deleting port 19661738-71a9-4fae-b298-282ddccb47a0 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:19.273080 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.392475 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.392709 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.517335 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Calling delete_port for 19661738-71a9-4fae-b298-282ddccb47a0 owned by network:floatingip {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:19.705536 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.709819 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-9c8901aa-8f6d-41ea-8684-705ad9cf9aad tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:19.771953 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'Port': ['ID=19661738-71a9-4fae-b298-282ddccb47a0,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:19.776060 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.066s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:19.886762 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for floatingip, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.957567 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.notifiers.nova.Notifier._send_nova_notification-16726521'] for floatingip, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:19.958576 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-77dfd5cb-c533-43d6-b74a-2cc4ae94d124 tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "DELETE /v2.0/floatingips/391d4545-674a-434a-9d8e-d2f524b79a84 HTTP/1.1" status: 204 len: 168 time: 1.0218096 May 25 16:17:21.506366 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52018) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:21.580395 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:21.611817 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:21.757154 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:21.762567 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-db7e5ef5-7839-4f6b-871b-c54bcb85ab38 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:21.778457 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Pushing event deleted for resources: {'SecurityGroup': ['ID=dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:21.781545 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:21.870896 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] Notify callbacks [] for security_group, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:21.872047 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-5abdf59a-7af6-4c76-bdca-e7a331a1005f tempest-TestVolumeBootPattern-943387186 tempest-TestVolumeBootPattern-943387186-project] 10.222.0.82 "DELETE /v2.0/security-groups/dc3f5a2f-20f5-49bc-8096-fa8c3abb8d64 HTTP/1.1" status: 204 len: 168 time: 0.3646722 May 25 16:17:22.010352 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52024) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:22.786055 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:23.341072 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Deleting port 921d6ca2-8279-4d74-8ee2-789091baf141 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:23.341409 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:23.488823 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] For port 921d6ca2-8279-4d74-8ee2-789091baf141, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=921d6ca2-8279-4d74-8ee2-789091baf141,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:23.489306 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:23.681641 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Calling delete_port for 921d6ca2-8279-4d74-8ee2-789091baf141 owned by network:router_interface {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:23.926443 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:23.930705 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-3b2fc6b8-1036-45d1-91ff-a82b56efc1be tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:23.976663 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Pushing event deleted for resources: {'Port': ['ID=921d6ca2-8279-4d74-8ee2-789091baf141,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:23.980794 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.050s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:24.259704 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:24.623756 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:17:24.917256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:17:24.925983 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Sync routers for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:17:24.930108 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] 10.222.0.82 "PUT /v2.0/routers/f8e5a29f-4d37-4854-88b3-cfab3a35cd5c/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.9183943 May 25 16:17:24.934607 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52030) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:24.937174 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:24.938313 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:25.139493 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: dv-cinder-786384-6-iscsi {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:25.142013 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks [] for router, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:25.333279 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:25.463272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220'}], 'enable_snat': True}, 'gw_port_id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:17:23Z', 'revision_number': 5, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port': {'id': '9c917315-3cdf-4b24-9114-5a5f1d99c173', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:0a:45:3e', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'device_owner': 'network:router_gateway', 'standard_attr_id': 515, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.220', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:41Z', 'updated_at': '2021-05-25T16:09:48Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:25.467136 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Checking router: f8e5a29f-4d37-4854-88b3-cfab3a35cd5c for host: dv-cinder-786384-6-iscsi {{(pid=99546) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:17:25.471657 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Deleting port 9c917315-3cdf-4b24-9114-5a5f1d99c173 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:25.472008 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:25.475625 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Routers {'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99546) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:25.476272 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'], 'active': True} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:25.601853 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] For port 9c917315-3cdf-4b24-9114-5a5f1d99c173, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=9c917315-3cdf-4b24-9114-5a5f1d99c173,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:25.602195 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:25.631011 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvr_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] Set router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c gateway port host: None {{(pid=99546) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:25.726144 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Calling delete_port for 9c917315-3cdf-4b24-9114-5a5f1d99c173 owned by network:router_gateway {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:25.750430 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_hamode_db [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:17:25Z', 'revision_number': 6, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99546) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:25.757116 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-3649a1d6-28b6-4d06-8582-741cf1aaaabe None None] The sync data for ids ['f8e5a29f-4d37-4854-88b3-cfab3a35cd5c'] in dv-cinder-786384-6-iscsi is: [{'id': 'f8e5a29f-4d37-4854-88b3-cfab3a35cd5c', 'name': 'tempest-TestVolumeBootPattern-830756281-router', 'tenant_id': 'a1e31a264b7949bea6aa47cb7781106b', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:40Z', 'updated_at': '2021-05-25T16:17:25Z', 'revision_number': 6, 'project_id': 'a1e31a264b7949bea6aa47cb7781106b', 'gw_port_host': None}] {{(pid=99546) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:17:25.962172 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:25.966107 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-01c136db-f2dd-454f-b21c-dd89296a59b5 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:26.011153 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Pushing event deleted for resources: {'Port': ['ID=9c917315-3cdf-4b24-9114-5a5f1d99c173,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:26.039621 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.073s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:26.104018 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.399866 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.403190 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.517200 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.520648 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Fanout notify agent at l3_agent the message router_deleted on router f8e5a29f-4d37-4854-88b3-cfab3a35cd5c {{(pid=99545) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:17:26.533697 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9c917315-3cdf-4b24-9114-5a5f1d99c173 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:26.534283 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] Notify callbacks [] for router, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.535097 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-255eac68-76e4-43e8-8159-0df11347bba6 admin admin] 10.222.0.82 "DELETE /v2.0/routers/f8e5a29f-4d37-4854-88b3-cfab3a35cd5c HTTP/1.1" status: 204 len: 168 time: 1.5994837 May 25 16:17:26.537645 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52034) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:26.581086 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 9c917315-3cdf-4b24-9114-5a5f1d99c173 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:26.581501 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 9c917315-3cdf-4b24-9114-5a5f1d99c173 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:26.581939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 921d6ca2-8279-4d74-8ee2-789091baf141 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:26.603504 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.db_base_plugin_v2 [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Deleting subnet a4d8980e-d38c-4165-af50-ed2f39dc708c {{(pid=99545) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:17:26.607690 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.619641 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 921d6ca2-8279-4d74-8ee2-789091baf141 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:26.620305 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 921d6ca2-8279-4d74-8ee2-789091baf141 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:26.665462 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-1746645'] for subnet, precommit_delete_associations {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:26.767132 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.134499 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-1892895', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-2319271'] for subnet, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.138595 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [-] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:27.174857 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Pushing event deleted for resources: {'Subnet': ['ID=a4d8980e-d38c-4165-af50-ed2f39dc708c,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:27.178213 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.039s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:27.179277 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for subnet, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.186138 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-857550c0-fb4f-478f-a470-8de792f2c1a8 admin admin] 10.222.0.82 "DELETE /v2.0/subnets/a4d8980e-d38c-4165-af50-ed2f39dc708c HTTP/1.1" status: 204 len: 168 time: 0.6473110 May 25 16:17:27.189066 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52038) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:27.192825 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-4a03ded1-3ef6-4492-9aa8-afee4ff357be None None] Network 6c61cff6-814b-4af0-8599-e026b705cc35 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:17:27.302149 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.374301 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Deleting port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:27.374649 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.460127 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] DHCP port deletion for 6c61cff6-814b-4af0-8599-e026b705cc35 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:17:27.485312 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] For port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:27.485757 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.528063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Deleting port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:27.528443 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.615225 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Calling delete_port for 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 owned by network:dhcp {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:27.689358 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] For port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,segment=NetworkSegment(bd457928-332b-49b5-8448-df2758dfb4c0),segment_id=bd457928-332b-49b5-8448-df2758dfb4c0)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:27.689358 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.774188 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.778770 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-802dec20-ebc3-46d4-acf0-6495b2aabb5f admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:27.809276 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Calling delete_port for 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:27.824239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Pushing event deleted for resources: {'Port': ['ID=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:27.827785 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.049s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:27.834730 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.db.l3_dvrscheduler_db [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] No DVR routers for this DVR port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 on host dv-cinder-786384-6-iscsi {{(pid=99545) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:27.871063 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:27.878401 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-d7ba9640-e599-41ae-a131-a03b6adb63b3 None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:27.958034 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Pushing event deleted for resources: {'Port': ['ID=6d8cf16c-bdd9-414f-a6e4-12cff843fb64,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:27.965346 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] No DVR routers for this DVR port 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:27.972532 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type AddressGroup changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.972878 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type ConntrackHelper changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.973228 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type Log changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.973564 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type Network changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.973893 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type Port changed None to 1.7 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.974224 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type PortForwarding changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.974553 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type QosPolicy changed None to 1.8 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.974895 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type SecurityGroup changed None to 1.4 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.975235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type SecurityGroupRule changed None to 1.2 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.975561 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type SubPort changed None to 1.0 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.975939 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type Subnet changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.976266 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.callbacks.version_manager [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Version for resource type Trunk changed None to 1.1 on consumer Open vSwitch agent@dv-cinder-786384-6-iscsi {{(pid=99546) _set_version /opt/stack/neutron/neutron/api/rpc/callbacks/version_manager.py:113}} May 25 16:17:27.981752 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-f484b690-7131-4f5e-b615-2963cbe5f4ae None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.104s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:28.041597 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.059509 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['6c61cff6-814b-4af0-8599-e026b705cc35']}} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:28.073430 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, 'bd457928-332b-49b5-8448-df2758dfb4c0') {'for_net_delete': True} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:28.073732 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.services.segments.db [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, 'bd457928-332b-49b5-8448-df2758dfb4c0') {} {{(pid=99545) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:28.088239 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-1746654'] for segment, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.105852 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.112631 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Releasing vlan 902 on physical network physnet1 to pool {{(pid=99545) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:17:28.120784 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-1923001'] for segment, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: Traceback (most recent call last): May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: timer() May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: cb(*args, **kw) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: return func(*args, **kwargs) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: self._notify() May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: self.callback(batched_events) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: event.method(event) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: return f(self, *a, **k) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._get(url).json() May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: return self.request(url, 'GET', **kwargs) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:17:28.127177 dv-cinder-786384-6-iscsi neutron-server[99545]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:17:28.250187 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:28.285928 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:28.286280 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d8cf16c-bdd9-414f-a6e4-12cff843fb64 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:28.298256 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.303079 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:28.306674 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-31fbff67-6806-4b87-a4b2-1b471f838556 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:28.331063 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Pushing event deleted for resources: {'Network': ['ID=6c61cff6-814b-4af0-8599-e026b705cc35,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:28.333609 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.027s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:28.423830 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-6efc1d34-adb8-40c9-abd8-679f944168f9 admin admin] 10.222.0.82 "DELETE /v2.0/networks/6c61cff6-814b-4af0-8599-e026b705cc35 HTTP/1.1" status: 204 len: 168 time: 1.2336669 May 25 16:17:28.426777 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.wsgi [-] (99545) accepted ('10.222.0.82', 52042) {{(pid=99545) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:29.075291 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cache_related_dvr_routers_info_before_interface_removal-491314'] for router_interface, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:29.504244 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Deleting port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 {{(pid=99545) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:29.505011 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:29.628748 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.db [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] For port 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99545) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:29.629093 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:29.756714 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.plugins.ml2.plugin [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Calling delete_port for 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 owned by network:router_interface {{(pid=99545) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:29.994321 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-1923052'] for port, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:29.998410 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-43cbd086-8a24-4561-9784-9d088c70be45 tempest-TestMinimumBasicScenario-1083600863 tempest-TestMinimumBasicScenario-1083600863-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:30.048511 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Pushing event deleted for resources: {'Port': ['ID=6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0,revision_number=None']} {{(pid=99545) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:30.081380 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG oslo_concurrency.lockutils [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.083s {{(pid=99545) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:30.440452 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron_lib.callbacks.manager [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._after_router_interface_deleted-16267247', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._cleanup_after_interface_removal-489890'] for router_interface, after_delete {{(pid=99545) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:30.810985 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.scheduler.l3_agent_scheduler [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Router dc208968-f643-4678-bf5f-5f1da544823d has already been hosted by L3 agent 665a025e-77a6-47e8-affe-189e03ba34d3 {{(pid=99545) _get_candidates /opt/stack/neutron/neutron/scheduler/l3_agent_scheduler.py:127}} May 25 16:17:31.015975 dv-cinder-786384-6-iscsi neutron-server[99545]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] Notify agent at l3_agent.dv-cinder-786384-6-iscsi the message routers_updated {{(pid=99545) _agent_notification /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:68}} May 25 16:17:31.025153 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Sync routers for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:125}} May 25 16:17:31.027424 dv-cinder-786384-6-iscsi neutron-server[99545]: INFO neutron.wsgi [None req-1859bfe4-8477-405a-9859-b4335ebdd210 admin admin] 10.222.0.82 "PUT /v2.0/routers/dc208968-f643-4678-bf5f-5f1da544823d/remove_router_interface HTTP/1.1" status: 200 len: 503 time: 2.5987182 May 25 16:17:31.029471 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52046) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:31.039104 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:31.039805 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:31.220062 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks [] for router, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:31.224230 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: dv-cinder-786384-6-iscsi {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:31.402348 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks [] for router_gateway, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:31.520462 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': {'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'external_fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130'}], 'enable_snat': True}, 'gw_port_id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:17:29Z', 'revision_number': 5, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port': {'id': '00083874-f8bf-4dc1-b252-9739e23df60e', 'name': '', 'network_id': '0c75ab82-64ae-4eef-9e41-0a0bc1e6ecdc', 'tenant_id': '', 'mac_address': 'fa:16:3e:6b:89:85', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'device_owner': 'network:router_gateway', 'standard_attr_id': 527, 'fixed_ips': [{'subnet_id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'ip_address': '172.27.2.130', 'prefixlen': 24}], 'allowed_address_pairs': [], 'extra_dhcp_opts': [], 'security_groups': [], 'description': '', 'binding:vnic_type': 'normal', 'binding:profile': {}, 'binding:host_id': 'dv-cinder-786384-6-iscsi', 'binding:vif_type': 'ovs', 'binding:vif_details': {'connectivity': 'l2', 'port_filter': True, 'ovs_hybrid_plug': False, 'datapath_type': 'system', 'bridge_name': 'br-int'}, 'port_security_enabled': False, 'tags': [], 'created_at': '2021-05-25T16:09:53Z', 'updated_at': '2021-05-25T16:10:00Z', 'revision_number': 4, 'project_id': '', 'subnets': [{'id': '9237fe36-57f4-4495-8b6e-e6960468c84c', 'cidr': '172.27.2.0/24', 'gateway_ip': '172.27.2.1', 'dns_nameservers': [], 'ipv6_ra_mode': None, 'subnetpool_id': None}], 'extra_subnets': [], 'address_scopes': {4: None, 6: None}, 'mtu': 1500}, 'enable_snat': True, 'gw_port_host': 'dv-cinder-786384-6-iscsi'}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:31.524275 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Checking router: dc208968-f643-4678-bf5f-5f1da544823d for host: dv-cinder-786384-6-iscsi {{(pid=99547) _ensure_host_set_on_ports /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:155}} May 25 16:17:31.533170 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvrscheduler_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Routers {'dc208968-f643-4678-bf5f-5f1da544823d'} bound to L3 agent in host dv-cinder-786384-6-iscsi {{(pid=99547) _get_router_ids_for_agent /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:422}} May 25 16:17:31.533837 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method get_ha_sync_data_for_host called with arguments (, 'dv-cinder-786384-6-iscsi', Agent(admin_state_up=True,agent_type='L3 agent',availability_zone='nova',binary='neutron-l3-agent',configurations={"agent_mode": "legacy", "ex_gw_ports": 3, "extensions": [], "floating_ips": 1, "handle_internal_only_routers": true, "interface_driver": "openvswitch", "interfaces": 3, "log_agent_heartbeats": false, "routers": 3},created_at=2021-05-25T15:54:38Z,description=,heartbeat_timestamp=2021-05-25T16:17:08Z,host='dv-cinder-786384-6-iscsi',id=665a025e-77a6-47e8-affe-189e03ba34d3,load=0,resource_versions=,resources_synced=,started_at=2021-05-25T15:54:39Z,topic='l3_agent')) {'router_ids': ['dc208968-f643-4678-bf5f-5f1da544823d'], 'active': True} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:31.597912 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Deleting port 00083874-f8bf-4dc1-b252-9739e23df60e {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:31.598254 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:31.695036 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_dvr_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] Set router dc208968-f643-4678-bf5f-5f1da544823d gateway port host: None {{(pid=99547) _build_routers_list /opt/stack/neutron/neutron/db/l3_dvr_db.py:820}} May 25 16:17:31.722469 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] For port 00083874-f8bf-4dc1-b252-9739e23df60e, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=00083874-f8bf-4dc1-b252-9739e23df60e,segment=NetworkSegment(1a95fb4a-3292-44a5-abc6-7330179a94c6),segment_id=1a95fb4a-3292-44a5-abc6-7330179a94c6)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:31.722941 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:31.845607 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.l3_hamode_db [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] neutron.services.l3_router.l3_router_plugin.L3RouterPlugin method _process_sync_ha_data called with arguments (, [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:17:31Z', 'revision_number': 6, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port_host': None}], 'dv-cinder-786384-6-iscsi', False) {} {{(pid=99547) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:31.849820 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.api.rpc.handlers.l3_rpc [None req-71200a34-8f19-4f4a-aa8b-64c7b46c94bd None None] The sync data for ids ['dc208968-f643-4678-bf5f-5f1da544823d'] in dv-cinder-786384-6-iscsi is: [{'id': 'dc208968-f643-4678-bf5f-5f1da544823d', 'name': 'tempest-TestVolumeBootPattern-1294247851-router', 'tenant_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'admin_state_up': True, 'status': 'ACTIVE', 'external_gateway_info': None, 'gw_port_id': None, 'description': '', 'availability_zones': ['nova'], 'distributed': False, 'ha': False, 'ha_vr_id': 0, 'availability_zone_hints': [], 'routes': [], 'flavor_id': None, 'tags': [], 'created_at': '2021-05-25T16:09:52Z', 'updated_at': '2021-05-25T16:17:31Z', 'revision_number': 6, 'project_id': 'cdb3487ec5004d52a006a7780a5c92ea', 'gw_port_host': None}] {{(pid=99547) sync_routers /opt/stack/neutron/neutron/api/rpc/handlers/l3_rpc.py:137}} May 25 16:17:32.016658 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Calling delete_port for 00083874-f8bf-4dc1-b252-9739e23df60e owned by network:router_gateway {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:32.213809 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:32.255984 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:32.256346 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 6d5f16c4-fb18-4012-92c1-ed5d64ca9aa0 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:32.265509 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.272965 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-23cb3c37-68d4-46c8-aeb6-1da876f426d5 service neutron] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:32.320079 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Pushing event deleted for resources: {'Port': ['ID=00083874-f8bf-4dc1-b252-9739e23df60e,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:32.356565 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.087s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:32.423121 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler._delete_dvr_internal_ports-490543'] for router_gateway, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.614959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._release_router_vr_id-489122', 'neutron.services.l3_router.service_providers.driver_controller.DriverController._clear_router_provider-2403'] for router, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.614959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks [] for router_controller, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.743799 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_HA_NAT_db_mixin._cleanup_ha_network-489630'] for router, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.744615 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.agentnotifiers.l3_rpc_agent_api [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Fanout notify agent at l3_agent the message router_deleted on router dc208968-f643-4678-bf5f-5f1da544823d {{(pid=99544) _notification_fanout /opt/stack/neutron/neutron/api/rpc/agentnotifiers/l3_rpc_agent_api.py:118}} May 25 16:17:32.764713 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] Notify callbacks [] for router, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.766073 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-19e9682a-7d23-4159-9336-e75388900c0b admin admin] 10.222.0.82 "DELETE /v2.0/routers/dc208968-f643-4678-bf5f-5f1da544823d HTTP/1.1" status: 204 len: 168 time: 1.7356215 May 25 16:17:32.768609 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52050) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:32.835343 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.db_base_plugin_v2 [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Deleting subnet 1e6dd188-e084-4f58-b243-e638c41a8cc8 {{(pid=99544) delete_subnet /opt/stack/neutron/neutron/db/db_base_plugin_v2.py:1043}} May 25 16:17:32.840315 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Notify callbacks [] for subnet, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.897260 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Notify callbacks ['neutron.services.segments.plugin.Plugin._validate_auto_address_subnet_delete-3244650'] for subnet, precommit_delete_associations {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:32.992181 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_precommit_handler-16165597'] for subnet, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.328434 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._subnet_delete_after_delete_handler-16166760', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123237', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_subnet_deleted-3108376', 'neutron.services.segments.plugin.SegmentHostRoutes.host_routes_after_delete-3108049'] for subnet, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.332596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-6f657bd5-b591-4812-81ed-05dc34b4d921 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:33.373124 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Pushing event deleted for resources: {'Subnet': ['ID=1e6dd188-e084-4f58-b243-e638c41a8cc8,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:33.376185 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.044s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:33.377177 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] Notify callbacks [] for subnet, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.382596 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-f344e6e5-f574-49e3-b689-97f2daf70dae None None] Network 2df16f06-2058-4b69-828c-2c380456e335 requested from dv-cinder-786384-6-iscsi {{(pid=99546) get_network_info /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:218}} May 25 16:17:33.383838 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-11a82ef9-9580-4fdc-a904-6d533bea965b admin admin] 10.222.0.82 "DELETE /v2.0/subnets/1e6dd188-e084-4f58-b243-e638c41a8cc8 HTTP/1.1" status: 204 len: 168 time: 0.6142149 May 25 16:17:33.386415 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52054) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:33.507943 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.External_net_db_mixin._before_network_delete_handler-16171945'] for network, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.594465 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Deleting port d0973574-d7d4-472e-a87f-448d82cb5ed3 {{(pid=99544) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:33.594747 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.654595 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.dhcp_rpc [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] DHCP port deletion for 2df16f06-2058-4b69-828c-2c380456e335 request from dv-cinder-786384-6-iscsi {{(pid=99546) release_dhcp_port /opt/stack/neutron/neutron/api/rpc/handlers/dhcp_rpc.py:266}} May 25 16:17:33.701766 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.db [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] For port d0973574-d7d4-472e-a87f-448d82cb5ed3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d0973574-d7d4-472e-a87f-448d82cb5ed3,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99544) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:33.702116 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.724733 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Deleting port d0973574-d7d4-472e-a87f-448d82cb5ed3 {{(pid=99546) _pre_delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1897}} May 25 16:17:33.725117 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Notify callbacks ['neutron.db.l3_db.L3_NAT_dbonly_mixin._prevent_l3_port_delete_callback-8747111865411'] for port, before_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.829619 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.db [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] For port d0973574-d7d4-472e-a87f-448d82cb5ed3, host dv-cinder-786384-6-iscsi, got binding levels [PortBindingLevel(driver='openvswitch',host='dv-cinder-786384-6-iscsi',level=0,port_id=d0973574-d7d4-472e-a87f-448d82cb5ed3,segment=NetworkSegment(1afdd637-4008-4148-a334-b60773d99ad1),segment_id=1afdd637-4008-4148-a334-b60773d99ad1)] {{(pid=99546) get_binding_level_objs /opt/stack/neutron/neutron/plugins/ml2/db.py:75}} May 25 16:17:33.830235 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Notify callbacks ['neutron.services.l3_router.l3_router_plugin.L3_NAT_dbonly_mixin._precommit_delete_port_callback-483811'] for port, precommit_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:33.838663 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.plugin [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Calling delete_port for d0973574-d7d4-472e-a87f-448d82cb5ed3 owned by network:dhcp {{(pid=99544) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:33.978811 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Calling delete_port for d0973574-d7d4-472e-a87f-448d82cb5ed3 owned by network:dhcp {{(pid=99546) delete_port /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1970}} May 25 16:17:34.014576 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_port_created_or_deleted-3070955'] for port, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.021900 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-af5a696f-e8c5-4353-9fa7-f8a74691fb38 tempest-AttachVolumeTestJSON-730703762 tempest-AttachVolumeTestJSON-730703762-project] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:34.030387 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Notify callbacks ['neutron.plugins.ml2.plugin.SecurityGroupServerNotifierRpcMixin.notify_sg_on_port_change-16140998', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-123231', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification-16267261', 'neutron.db.l3_db.L3RpcNotifierMixin._notify_routers_callback-8747111867558', 'neutron.db.l3_dvrscheduler_db._notify_port_delete-8747111529912'] for port, after_delete {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.030387 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.001s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:34.072187 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Pushing event deleted for resources: {'Port': ['ID=d0973574-d7d4-472e-a87f-448d82cb5ed3,revision_number=None']} {{(pid=99546) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:34.076253 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG oslo_concurrency.lockutils [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.048s {{(pid=99546) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:34.081103 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.db.l3_dvrscheduler_db [None req-e5124824-e3e8-48ae-b6d9-6a44b14bd111 None None] No DVR routers for this DVR port d0973574-d7d4-472e-a87f-448d82cb5ed3 on host dv-cinder-786384-6-iscsi {{(pid=99546) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:34.180935 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Pushing event deleted for resources: {'Port': ['ID=d0973574-d7d4-472e-a87f-448d82cb5ed3,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:34.183515 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.db.l3_dvrscheduler_db [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] No DVR routers for this DVR port d0973574-d7d4-472e-a87f-448d82cb5ed3 on host dv-cinder-786384-6-iscsi {{(pid=99544) get_dvr_routers_to_remove /opt/stack/neutron/neutron/db/l3_dvrscheduler_db.py:214}} May 25 16:17:34.185127 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.163s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:34.265777 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 00083874-f8bf-4dc1-b252-9739e23df60e no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:34.307549 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_precommit_handler-16158802', 'neutron.services.segments.db._delete_segments_for_network-8747111927516'] for network, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.310483 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: 00083874-f8bf-4dc1-b252-9739e23df60e {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:34.310838 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device 00083874-f8bf-4dc1-b252-9739e23df60e not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:34.311286 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d0973574-d7d4-472e-a87f-448d82cb5ed3 no longer exists at agent ovs-agent-dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:243}} May 25 16:17:34.321533 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] neutron.services.segments.plugin.Plugin method get_segments called with arguments (,) {'filters': {'network_id': ['2df16f06-2058-4b69-828c-2c380456e335']}} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:34.336906 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] neutron.services.segments.plugin.Plugin method delete_segment called with arguments (, '1afdd637-4008-4148-a334-b60773d99ad1') {'for_net_delete': True} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:34.337596 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.services.segments.db [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] neutron.services.segments.plugin.Plugin method get_segment called with arguments (, '1afdd637-4008-4148-a334-b60773d99ad1') {} {{(pid=99544) wrapper /usr/local/lib/python3.8/dist-packages/oslo_log/helpers.py:65}} May 25 16:17:34.350938 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.db._prevent_segment_delete_with_port_bound-8747111936925', 'neutron.services.segments.plugin.Plugin._prevent_segment_delete_with_subnet_associated-3244641'] for segment, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.351597 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.plugin [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] No Port match for: d0973574-d7d4-472e-a87f-448d82cb5ed3 {{(pid=99546) port_bound_to_host /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2224}} May 25 16:17:34.351953 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron.plugins.ml2.rpc [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Device d0973574-d7d4-472e-a87f-448d82cb5ed3 not bound to the agent host dv-cinder-786384-6-iscsi {{(pid=99546) update_device_down /opt/stack/neutron/neutron/plugins/ml2/rpc.py:251}} May 25 16:17:34.374923 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986'] for segment, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.382038 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.plugins.ml2.drivers.type_vlan [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Releasing vlan 919 on physical network physnet1 to pool {{(pid=99544) release_segment /opt/stack/neutron/neutron/plugins/ml2/drivers/type_vlan.py:304}} May 25 16:17:34.382438 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._handle_segment_change-16170986', 'neutron.services.segments.plugin.NovaSegmentNotifier._notify_segment_deleted-3070910'] for segment, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: Traceback (most recent call last): May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/hub.py", line 476, in fire_timers May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: timer() May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/eventlet/hubs/timer.py", line 59, in __call__ May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: cb(*args, **kw) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/common/utils.py", line 905, in wrapper May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: return func(*args, **kwargs) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 58, in synced_send May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: self._notify() May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/notifiers/batch_notifier.py", line 69, in _notify May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: self.callback(batched_events) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 211, in _send_notifications May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: event.method(event) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 379, in _delete_nova_inventory May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_id = self._get_aggregate_id(event.segment_id) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/opt/stack/neutron/neutron/services/segments/plugin.py", line 366, in _get_aggregate_id May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: aggregate_uuid = self.p_client.list_aggregates( May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 57, in wrapper May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: return f(self, *a, **k) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 553, in list_aggregates May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._get(url).json() May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/neutron_lib/placement/client.py", line 189, in _get May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: return self._client.get(url, endpoint_filter=self._ks_filter, May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1141, in get May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: return self.request(url, 'GET', **kwargs) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 811, in request May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: base_url = self.get_endpoint(auth, allow=allow, May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1241, in get_endpoint May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: auth = self._auth_required(auth, 'determine endpoint URL') May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: File "/usr/local/lib/python3.8/dist-packages/keystoneauth1/session.py", line 1181, in _auth_required May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: raise exceptions.MissingAuthPlugin(msg_fmt % msg) May 25 16:17:34.396596 dv-cinder-786384-6-iscsi neutron-server[99544]: keystoneauth1.exceptions.auth_plugins.MissingAuthPlugin: An auth plugin is required to determine endpoint URL May 25 16:17:34.528228 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._network_delete_after_delete_handler-16158811', 'neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_payload_event-16304695', 'neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._native_event_send_dhcp_notification_payload-16267238', 'neutron.services.l3_router.l3_router_plugin.DVRResourceOperationHandler.delete_fip_namespaces_for_ext_net-490546'] for network, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.532081 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-300ff0c4-5fa3-4e31-8c8c-53f1db7d7527 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:34.546099 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Notify callbacks ['neutron.api.rpc.agentnotifiers.dhcp_rpc_agent_api.DhcpAgentNotifyAPI._send_dhcp_notification-16267203'] for network, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:34.556745 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Pushing event deleted for resources: {'Network': ['ID=2df16f06-2058-4b69-828c-2c380456e335,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:34.559875 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.028s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:34.562280 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-09a155ca-7ee9-4a5c-99a9-5f15baeb1724 admin admin] 10.222.0.82 "DELETE /v2.0/networks/2df16f06-2058-4b69-828c-2c380456e335 HTTP/1.1" status: 204 len: 168 time: 1.1744068 May 25 16:17:34.942959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52060) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:34.991202 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-92213f0f-6a0b-4b23-9907-c4d6b11e8d4e admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=a1e31a264b7949bea6aa47cb7781106b&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0469894 May 25 16:17:34.993526 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52064) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:35.059394 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:35.089497 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:35.322227 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:35.326825 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-d479150b-e2d9-4247-ba31-98c0b9ab1ba8 admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:35.341699 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=41f65755-9a1f-492d-8410-648d24ee9269,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:35.345926 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.019s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:35.349409 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:35.350588 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9b040c50-eab2-40b5-b46c-73c93baaa5ed admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/41f65755-9a1f-492d-8410-648d24ee9269 HTTP/1.1" status: 204 len: 168 time: 0.3561606 May 25 16:17:35.670723 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-5628872b-c049-486a-867e-59ef6e5a7ac1 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:36.154767 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52072) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:36.208171 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-9e441321-fa2a-40b4-9ec1-20055aa500b1 admin admin] 10.222.0.82 "GET /v2.0/security-groups?tenant_id=cdb3487ec5004d52a006a7780a5c92ea&name=default HTTP/1.1" status: 200 len: 2979 time: 0.0520411 May 25 16:17:36.212781 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [-] (99544) accepted ('10.222.0.82', 52076) {{(pid=99544) server /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:992}} May 25 16:17:36.273647 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Notify callbacks [] for security_group, before_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:36.301161 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Notify callbacks [] for security_group, precommit_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:36.436597 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Notify callbacks ['neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.handle_event-16304712'] for security_group, after_delete {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:36.440614 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-959ee9c7-9f4c-4e2a-965c-4f60f32918fd admin admin] Lock "event-dispatch" acquired by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: waited 0.000s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:355}} May 25 16:17:36.453654 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.api.rpc.handlers.resources_rpc [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Pushing event deleted for resources: {'SecurityGroup': ['ID=f168a92e-dc40-4e0c-973d-75bb5df77ee1,revision_number=None']} {{(pid=99544) push /opt/stack/neutron/neutron/api/rpc/handlers/resources_rpc.py:237}} May 25 16:17:36.457506 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG oslo_concurrency.lockutils [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Lock "event-dispatch" released by "neutron.plugins.ml2.ovo_rpc._ObjectChangeHandler.dispatch_events" :: held 0.017s {{(pid=99544) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:367}} May 25 16:17:36.461329 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron_lib.callbacks.manager [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] Notify callbacks [] for security_group, before_response {{(pid=99544) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:36.462433 dv-cinder-786384-6-iscsi neutron-server[99544]: INFO neutron.wsgi [None req-1dc07e86-047b-45aa-8139-083dea433a1f admin admin] 10.222.0.82 "DELETE /v2.0/security-groups/f168a92e-dc40-4e0c-973d-75bb5df77ee1 HTTP/1.1" status: 204 len: 168 time: 0.2508817 May 25 16:17:38.200085 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:38.408487 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-e42c4e48-f849-46c5-b388-0a2bb2a1f2e2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:39.145148 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:40.602513 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:41.188638 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:17:44.299223 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:18:05.693414 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-30b6812b-850f-4d6c-8acd-ebf5ac8a3aa5 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:08.167502 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:08.459038 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2fdb1ad3-9d24-42fb-b64a-6ec2f55fa7b6 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:09.130047 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:10.613046 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:11.165018 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:21.299247 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:18:35.683590 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2e16be2f-dc2d-49a9-b128-a3fde2cab670 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:38.218645 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:38.443686 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-787616ff-9de4-4683-9bcc-e7dbb797eb8a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:39.144791 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:40.666126 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:41.204002 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:18:46.975520 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [req-e31b1655-d31d-4f5f-b868-8eb2c8119f89 req-ae1b9d3b-8863-4bf3-ab26-74327c963db5 service neutron] (99544) timed out ('10.222.0.42', 57064) {{(pid=99544) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 25 16:18:58.299343 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:19:05.711315 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-b2bd5c02-2472-4791-a214-ebb37293111a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:08.241616 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:08.511520 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-f9ca02a0-69b2-4e23-9deb-21cbf294c022 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:09.208529 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:10.646705 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:11.168506 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:35.298396 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:19:35.676215 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-8d8bfd18-f73f-4a9e-8961-63a2d2214081 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:38.195927 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:38.419080 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-dd8d937a-594b-4d08-9d0f-31eb873963cc None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:39.195870 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:40.618906 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:41.173026 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:19:43.836959 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [req-5bf8844b-52e2-465f-9881-a05632a60c2e req-53f88553-0794-475a-bc14-a7fa84bb973d service neutron] (99544) timed out ('10.222.0.42', 57057) {{(pid=99544) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 25 16:20:05.738165 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-222bafb5-0a83-4014-a867-2ba9ca413d66 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:08.183800 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:08.463992 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7a246f1e-2206-4343-8661-2a467813619a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:09.137958 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:10.692383 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:11.218233 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:12.297772 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:20:35.708617 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-54f0967a-5372-4943-a71a-8855998dc32c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:38.165575 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:38.434314 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-db14c5c2-b3dc-4f15-b0cb-9421d80d166e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:39.171109 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:40.673853 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:41.177424 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:20:49.297542 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:21:05.673385 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-97705236-4e7c-4d58-8f7d-a3cd9533c22e None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:08.165389 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:08.462232 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4d7444a8-895c-4876-a53b-7bd3fc5fdc33 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:09.170933 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:10.673825 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:11.211635 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:26.296844 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:21:35.673304 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-d9983d58-d7ea-4842-b853-2cd12d586b54 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:38.176230 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:38.421591 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-93be99c8-3b08-4051-8785-48e1af367d70 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:39.198233 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:40.644408 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:41.192858 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:21:42.683229 dv-cinder-786384-6-iscsi neutron-server[99544]: DEBUG neutron.wsgi [req-399eff4e-c039-45fe-ae9c-83aa280e5204 req-aee9ab32-3b94-47d7-ad6a-9a651fb18726 service neutron] (99544) timed out ('10.222.0.82', 45484) {{(pid=99544) process_request /usr/local/lib/python3.8/dist-packages/eventlet/wsgi.py:830}} May 25 16:22:03.295948 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:22:05.696495 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-3d6c4f17-7749-4580-b039-79a9b07013f8 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:08.175437 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:08.467511 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-cc8add0c-ab82-42c3-9aee-e4347c9d364a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:09.204408 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:10.670739 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:11.202795 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:35.677779 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-5c9a31a2-a14d-46c7-99c5-05f1209d4503 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:38.143909 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-4f413ad3-5122-453d-9f2c-965e0c33532a None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:38.483830 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-7235cb5b-8a6d-40b8-b4dd-2931b086f1c2 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:39.157632 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-2b2492d3-f336-4be0-9c3b-67593837fee7 None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:40.294501 dv-cinder-786384-6-iscsi neutron-server[99547]: DEBUG neutron.db.agents_db [None req-7ba4ee36-aa7e-4499-82ef-d9386746bcb6 None None] Agent healthcheck: found 6 active agents {{(pid=99547) agent_health_check /opt/stack/neutron/neutron/db/agents_db.py:314}} May 25 16:22:40.669718 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-98611f56-c288-4f87-bdcd-b51ffff0836c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}} May 25 16:22:41.263767 dv-cinder-786384-6-iscsi neutron-server[99546]: DEBUG neutron_lib.callbacks.manager [None req-9bdf9b1e-64e0-4128-af60-dc994a52389c None None] Notify callbacks ['neutron.services.segments.db._update_segment_host_mapping_for_agent-8747111927534', 'neutron.plugins.ml2.plugin.Ml2Plugin._retry_binding_revived_agents-16160156'] for agent, after_update {{(pid=99546) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:192}}